-- Logs begin at Mon 2022-09-19 15:33:37 UTC, end at Mon 2022-09-19 17:03:21 UTC. -- Sep 19 15:58:30.765437 dv-cinder-858370-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 19 15:58:30.774779 dv-cinder-858370-2-iscsi devstack@keystone.service[81602]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 19 15:58:30.778627 dv-cinder-858370-2-iscsi devstack@keystone.service[81602]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Sep 19 15:58:30 2022] *** Sep 19 15:58:30.778627 dv-cinder-858370-2-iscsi devstack@keystone.service[81602]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 19 15:58:30.778627 dv-cinder-858370-2-iscsi devstack@keystone.service[81602]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 19 15:58:30.778627 dv-cinder-858370-2-iscsi devstack@keystone.service[81602]: nodename: dv-cinder-858370-2-iscsi Sep 19 15:58:30.778627 dv-cinder-858370-2-iscsi devstack@keystone.service[81602]: machine: x86_64 Sep 19 15:58:30.778627 dv-cinder-858370-2-iscsi devstack@keystone.service[81602]: clock source: unix Sep 19 15:58:30.778627 dv-cinder-858370-2-iscsi devstack@keystone.service[81602]: pcre jit disabled Sep 19 15:58:30.778627 dv-cinder-858370-2-iscsi devstack@keystone.service[81602]: detected number of CPU cores: 4 Sep 19 15:58:30.778627 dv-cinder-858370-2-iscsi devstack@keystone.service[81602]: current working directory: / Sep 19 15:58:30.778627 dv-cinder-858370-2-iscsi devstack@keystone.service[81602]: detected binary path: /usr/bin/uwsgi-core Sep 19 15:58:30.779168 dv-cinder-858370-2-iscsi devstack@keystone.service[81602]: your processes number limit is 47680 Sep 19 15:58:30.779168 dv-cinder-858370-2-iscsi devstack@keystone.service[81602]: your memory page size is 4096 bytes Sep 19 15:58:30.779168 dv-cinder-858370-2-iscsi devstack@keystone.service[81602]: detected max file descriptor number: 2048 Sep 19 15:58:30.779168 dv-cinder-858370-2-iscsi devstack@keystone.service[81602]: lock engine: pthread robust mutexes Sep 19 15:58:30.779168 dv-cinder-858370-2-iscsi devstack@keystone.service[81602]: thunder lock: enabled Sep 19 15:58:30.779168 dv-cinder-858370-2-iscsi devstack@keystone.service[81602]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 19 15:58:30.779168 dv-cinder-858370-2-iscsi devstack@keystone.service[81602]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 19 15:58:30.818288 dv-cinder-858370-2-iscsi devstack@keystone.service[81602]: Python main interpreter initialized at 0x55a1f1d4a9c0 Sep 19 15:58:30.818288 dv-cinder-858370-2-iscsi devstack@keystone.service[81602]: python threads support enabled Sep 19 15:58:30.818288 dv-cinder-858370-2-iscsi devstack@keystone.service[81602]: your server socket listen backlog is limited to 100 connections Sep 19 15:58:30.818288 dv-cinder-858370-2-iscsi devstack@keystone.service[81602]: your mercy for graceful operations on workers is 90 seconds Sep 19 15:58:30.818706 dv-cinder-858370-2-iscsi devstack@keystone.service[81602]: mapped 403077 bytes (393 KB) for 2 cores Sep 19 15:58:30.818788 dv-cinder-858370-2-iscsi devstack@keystone.service[81602]: *** Operational MODE: preforking *** Sep 19 15:58:30.819003 dv-cinder-858370-2-iscsi devstack@keystone.service[81602]: *** uWSGI is running in multiple interpreter mode *** Sep 19 15:58:30.819003 dv-cinder-858370-2-iscsi devstack@keystone.service[81602]: spawned uWSGI master process (pid: 81602) Sep 19 15:58:30.819209 dv-cinder-858370-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 19 15:58:30.820545 dv-cinder-858370-2-iscsi devstack@keystone.service[81602]: spawned uWSGI worker 1 (pid: 81617, cores: 1) Sep 19 15:58:30.820545 dv-cinder-858370-2-iscsi devstack@keystone.service[81602]: spawned uWSGI worker 2 (pid: 81618, cores: 1) Sep 19 15:58:30.820545 dv-cinder-858370-2-iscsi devstack@keystone.service[81602]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 19 15:58:35.145413 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 19 15:58:35.145413 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: from cryptography.utils import int_from_bytes Sep 19 15:58:35.146170 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 19 15:58:35.146170 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: from cryptography.utils import int_from_bytes Sep 19 15:58:35.146729 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 19 15:58:35.146729 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: from cryptography.utils import int_from_bytes Sep 19 15:58:35.146863 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 19 15:58:35.146863 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: from cryptography.utils import int_from_bytes Sep 19 15:58:35.195477 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 19 15:58:35.195673 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 19 15:58:35.195771 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 19 15:58:35.195862 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 19 15:58:35.195943 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] command line args: [] {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 19 15:58:35.196036 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] command line args: [] {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 19 15:58:35.196119 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 19 15:58:35.196242 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 19 15:58:35.196327 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] ================================================================================ {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 19 15:58:35.196404 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] ================================================================================ {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 19 15:58:35.196503 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] admin_token = **** {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.196661 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] admin_token = **** {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.196753 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] config_dir = [] {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.196909 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] config_dir = [] {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.196994 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.197072 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] config_source = [] {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.197159 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.197261 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] debug = True {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.197391 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] config_source = [] {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.197468 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] debug = True {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.197536 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.197711 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.197810 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.197873 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.197984 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.198068 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.199624 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.199757 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.199878 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.199961 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.200078 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] list_limit = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.200160 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] list_limit = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.200282 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] log_config_append = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.200357 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] log_config_append = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.200484 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.200564 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.200680 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] log_dir = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.200759 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] log_dir = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.200829 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] log_file = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.200903 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] log_file = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.200983 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.201059 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.201174 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.201248 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.201361 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.201441 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.201523 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.201642 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.201718 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.201832 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.201906 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.202038 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.202113 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.202234 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.202309 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.202454 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.202581 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.202665 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.202780 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.202854 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.202966 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.203040 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.203151 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.203237 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.203314 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.203428 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.203502 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.203620 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.203711 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.203792 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.203891 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.57/identity {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.204579 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.57/identity {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.204686 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] publish_errors = False {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.204766 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] publish_errors = False {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.204836 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.204901 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.204968 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.205046 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.205116 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.205184 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.205251 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.205389 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.205474 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.205545 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.205614 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] standard_threads = False {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.205688 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] standard_threads = False {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.205763 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.205843 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.205919 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.205988 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.206104 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.206191 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.206262 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] use_journal = False {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.206396 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] use_journal = False {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.206472 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] use_json = False {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.206541 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] use_json = False {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.206620 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] use_stderr = False {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.206695 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] use_stderr = False {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.206771 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] use_syslog = False {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.206848 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] use_syslog = False {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.206924 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.206993 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 15:58:35.207061 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.207134 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.207210 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.207290 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.207408 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.207489 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.207592 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.207680 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.207841 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.207929 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.208077 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.208161 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.208326 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.208403 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.208564 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.208649 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.208807 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.208883 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.209054 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.209138 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.209303 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.209387 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.209535 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.209612 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.209779 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.209862 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.210011 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.210096 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.210255 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.210333 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.210524 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.210609 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.210760 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.210854 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.211013 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.211090 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.211265 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.211341 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.211508 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.211587 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.211743 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.211820 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.211993 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.212120 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.212251 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.212331 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.212504 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.212585 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.212748 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.212825 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.212978 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.213062 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.213205 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.213280 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.213433 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.213511 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.213667 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.213744 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.213909 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.213986 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.214154 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.214273 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.214464 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.214582 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.214696 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.214809 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.214923 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.215036 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.215157 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.215278 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.215392 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.215504 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.215622 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.215743 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.215865 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.215981 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.216094 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.216206 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.216318 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.216431 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.216551 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.216667 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.216786 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.216901 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.217014 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.217128 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.217315 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.217431 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.217675 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.217795 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.217908 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.218021 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.218145 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.218260 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.218388 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.218504 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.218628 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.218750 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.218872 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.218987 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.219107 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.219221 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.219334 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.219452 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.219577 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.219739 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.219839 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.219932 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.220025 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.220118 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.220233 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.220332 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.220426 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.220553 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.220679 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.220806 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.220921 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.221024 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.221136 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.221249 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.221374 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.221497 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.221612 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.221721 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.221841 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.221967 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.222132 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.222260 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.222423 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.222562 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.222685 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.222800 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.222913 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.223025 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.223138 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.223259 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.223375 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.223490 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.223603 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.223715 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.223828 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.223953 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.224059 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.224171 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.224284 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.224398 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.224511 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.224635 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.224742 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.224844 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.224956 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.225069 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.225181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.225305 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.225422 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.225542 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.225656 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.225769 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.225882 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.226004 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.226119 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.226239 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.226353 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.226499 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.226616 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.226738 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.226853 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.226967 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.227080 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.227194 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.227306 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.227427 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.227543 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.227657 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.227770 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.227883 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.227996 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.228118 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.228237 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.228351 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.228464 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.228585 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.228700 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.228886 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.229003 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.229116 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.229229 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.229342 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.229468 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.229593 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.229717 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.229832 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.229946 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.230059 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.230172 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.230284 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.230438 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.230557 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.230672 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.230786 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.230899 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.231012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.231134 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.231253 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.231368 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.231482 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.231596 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.231717 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.231844 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.231967 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.232352 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.232476 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.232591 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.232733 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.232860 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] auth.external = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.232976 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] auth.external = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.233100 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.233224 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.233341 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.233463 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.233580 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.233694 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.233818 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] auth.password = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.233934 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] auth.password = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.234049 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] auth.token = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.234163 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] auth.token = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.234277 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.234410 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.234540 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.234658 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.234772 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.234888 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.235002 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.235116 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.235240 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.235356 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.235471 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.235575 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.235688 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.235801 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.235928 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] credential.caching = True {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.236045 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] credential.caching = True {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.236159 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.236279 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.236395 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.236509 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.236632 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.236748 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.236862 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.236977 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.237091 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.237091 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.237321 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.237321 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.237536 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.237654 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.237768 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.237768 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.237992 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.237992 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.238208 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.238322 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.238466 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.238466 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.238692 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.238692 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.238915 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.238915 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.239132 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.239245 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.239368 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.239484 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] federation.caching = True {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.239603 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.239717 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.239831 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] federation.caching = True {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.239944 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.240067 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.240183 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.240297 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.240411 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.240531 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.240657 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.240774 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.240888 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.241002 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.241117 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.241231 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.241359 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.241475 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.241595 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.241711 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.241825 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.241938 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.242060 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.242186 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.242332 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] identity.caching = True {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.242496 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.242614 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.242728 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] identity.caching = True {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.242921 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.243040 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.243155 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.243271 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.243386 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.243500 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.243623 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.243745 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.243861 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.243975 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.244088 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.244202 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.244326 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.244442 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.244556 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.244670 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.244784 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.244898 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.245020 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.245136 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.245250 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.245366 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.245480 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.245594 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.245716 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.245832 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.245954 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.246070 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.246184 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.246298 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.246438 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.246557 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.246671 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.246785 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.246898 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.247012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.247184 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.247303 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.247418 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.247533 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.247647 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.247760 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.247882 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.247999 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.248113 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.248227 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.248341 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.248456 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.248579 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.248701 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.248816 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.248937 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.249053 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.249166 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.249289 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.249405 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.249520 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.249634 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.249748 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.249871 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.249987 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.250101 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.250215 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.250328 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.250452 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.250580 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.250697 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.251365 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.251545 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.251733 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.251926 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.252119 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.252303 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.252551 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.252743 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.252932 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.253121 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.253328 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.253521 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.253712 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.253907 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.254104 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.254104 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.254548 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.254548 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.254907 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.255098 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.255291 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.255291 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.255675 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.255675 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.256038 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.256231 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.256409 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.256532 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.256657 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.256773 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.256889 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.257004 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] ldap.user = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.257125 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.257240 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.257362 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] ldap.user = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.257478 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.257593 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.257708 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.257821 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.257937 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.258059 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.258175 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.258290 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.258420 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.258536 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.258651 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.258779 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.258897 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.259013 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.259128 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.259243 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.259357 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.259480 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.259597 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.259711 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.259826 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.259940 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.260055 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.260177 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.260292 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.260405 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.260519 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.260632 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.260746 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.260868 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.260983 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.261098 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.261212 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.261212 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.261212 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.261212 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.261212 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.261212 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.261837 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.261951 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.262065 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.262188 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.262304 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.262457 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.262574 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.262709 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.262709 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.262709 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.263047 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.263164 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.263164 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.263384 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.263499 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.263621 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.263736 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.263850 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.263964 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.264078 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.264193 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.264317 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.264433 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.264546 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.264660 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] resource.caching = True {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.264774 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.264889 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.265012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.265128 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.265128 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.265128 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.265128 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.265128 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.265128 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.265128 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.265856 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.265971 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.266091 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.266206 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] role.caching = True {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.266329 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.267080 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] role.driver = sql {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.267199 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.267314 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.267429 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.267556 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.267716 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.267833 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.267948 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.268061 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.268174 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] resource.caching = True {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.268366 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.268486 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.268600 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.268600 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.268600 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.268600 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.268600 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.268600 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.268600 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.268600 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.268600 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.269648 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.269648 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.269879 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.269993 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.270107 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.270220 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.270334 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.270487 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.270606 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.270721 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.270721 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.270935 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.270935 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.271158 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.271274 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.271274 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.271492 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] role.caching = True {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.271607 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.271607 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.271831 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] role.driver = sql {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.271954 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.271954 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.272169 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.272283 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.272283 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.272541 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.272660 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.272660 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.272873 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.272988 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.272988 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.273210 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.273327 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.273327 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] token.caching = True {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.273541 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.273657 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.273657 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.273879 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.273995 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.273995 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.274207 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.274321 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.274321 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.274648 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.274797 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.274797 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.275015 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.275129 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.275286 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.275402 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.275517 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.275631 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.275746 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.275989 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.276249 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.276511 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.276774 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.276905 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.277057 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.277175 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.277351 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] database.connection = **** {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.277471 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.277599 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.277867 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.278107 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.278328 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.278476 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.278632 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.278762 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.278907 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.279025 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.279140 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.279253 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.279403 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.279555 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.279674 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.279789 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.279904 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.280019 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.280133 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.280387 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.280645 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.280899 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.281161 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.281461 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.281675 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.281865 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.282028 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.282155 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81617) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 19 15:58:35.282259 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.282496 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.282790 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.283046 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.283337 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.283613 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.283923 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.285556 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.285828 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.286085 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.286445 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.286704 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] token.caching = True {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.286980 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.287264 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.287567 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.287879 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.288143 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.288448 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.288705 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.288960 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.289217 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.289462 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.289706 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.289948 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.290222 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.290568 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.290810 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.291317 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.291635 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.291947 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] database.connection = **** {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.292200 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.292499 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.292744 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.292992 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.293235 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.293556 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.293840 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.294104 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.294354 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.294667 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.294920 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.295161 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.295413 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.295654 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.295902 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.296134 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.296435 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.296675 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 15:58:35.296848 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81618) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 19 15:58:35.312529 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81617) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 15:58:35.312777 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81617) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 15:58:35.313010 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81617) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 15:58:35.322308 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81617) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 15:58:35.323297 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81618) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 15:58:35.323532 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81618) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 15:58:35.323758 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81618) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 15:58:35.333060 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81618) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 15:58:35.351709 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81617) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 15:58:35.351941 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81617) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 15:58:35.354109 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81618) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 15:58:35.354415 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81618) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 15:58:35.405735 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81617) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 15:58:35.406687 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81618) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 15:58:35.457345 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81617) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 15:58:35.457615 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81617) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 15:58:35.460089 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81618) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 15:58:35.460329 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81618) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 15:58:35.616027 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81617) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 15:58:35.616302 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81617) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 15:58:35.616441 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81617) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 15:58:35.616667 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81617) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 15:58:35.616881 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81617) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 15:58:35.617092 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81617) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 15:58:35.618120 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81618) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 15:58:35.618308 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81618) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 15:58:35.618567 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81618) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 15:58:35.618786 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81618) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 15:58:35.619007 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81618) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 15:58:35.619225 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81618) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 15:58:35.640266 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81617) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 15:58:35.640497 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81617) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 15:58:35.640677 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81617) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 15:58:35.640893 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81617) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 15:58:35.641042 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81618) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 15:58:35.641177 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81617) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 15:58:35.641318 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81618) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 15:58:35.641962 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81617) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 15:58:35.642097 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81618) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 15:58:35.642097 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81618) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 15:58:35.642357 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81617) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 15:58:35.642518 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81618) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 15:58:35.642518 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81618) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 15:58:35.642790 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81618) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 15:58:35.653830 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81618) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 15:58:35.654316 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81617) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 15:58:35.667540 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81618) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 15:58:35.667717 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81618) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 15:58:35.667802 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81617) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 15:58:35.667966 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81618) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 15:58:35.668108 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81617) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 15:58:35.668237 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81618) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 15:58:35.668365 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81617) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 15:58:35.668492 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81618) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 15:58:35.668617 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81617) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 15:58:35.668752 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81618) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 15:58:35.668879 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81617) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 15:58:35.669007 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81618) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 15:58:35.669145 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81617) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 15:58:35.669145 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81617) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 15:58:35.685438 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.685650 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 19 15:58:35.685797 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.685975 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.686067 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 19 15:58:35.686241 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 19 15:58:35.686412 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.686544 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.686671 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 19 15:58:35.686802 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.686954 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.687097 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.687227 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.687355 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.687485 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.687613 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.696771 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.697011 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.697144 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.697278 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.697410 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.697554 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.697695 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.697829 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.705037 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81618) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 15:58:35.705239 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81617) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 15:58:35.708029 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81618) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 15:58:35.708176 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81617) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 15:58:35.708307 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.708434 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.708588 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.708718 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.708858 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.709000 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.709134 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.709265 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.709391 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.709517 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.709657 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.709789 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.709917 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.710043 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.710169 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.710319 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.710488 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.710623 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.710754 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.710885 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.733679 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.733844 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.735191 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81617) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 15:58:35.735329 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81618) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 15:58:35.735490 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.735632 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.740000 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81618) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 15:58:35.740171 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81617) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 15:58:35.740303 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.740457 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.740666 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.740806 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.746996 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81618) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 15:58:35.747117 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81617) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 15:58:35.747300 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.747427 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.751243 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81618) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 15:58:35.751542 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.751830 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.753130 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81617) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 15:58:35.753430 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.753718 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.753931 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.754067 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.754232 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.754362 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.754553 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.754709 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.754842 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.754973 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.755104 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.755235 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.770875 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.771102 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.771248 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.771387 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.774924 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81617) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 15:58:35.775247 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81618) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 15:58:35.778069 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.778302 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.778462 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.778740 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.783023 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81617) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 15:58:35.783517 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81618) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 15:58:35.786270 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81617) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 15:58:35.786765 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81618) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 15:58:35.789498 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.789791 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.789920 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.790077 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.790203 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.790359 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.790515 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.790647 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.790816 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.790942 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.791358 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.791660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.806803 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81617) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 15:58:35.807091 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.807359 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.807486 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81618) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 15:58:35.807611 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.807747 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.807915 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.808183 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.814895 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.815444 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.816422 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.817098 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.817932 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.818208 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.818629 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.818907 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.820848 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81617) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 15:58:35.821133 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.821414 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.821564 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81618) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 15:58:35.821696 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.821858 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.822092 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.822174 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.822431 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.822726 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.833688 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81617) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 15:58:35.833974 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.834255 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.834463 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81618) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 15:58:35.834573 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.834756 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.834956 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.835036 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.835238 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.835320 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.835536 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.835617 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.835855 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.836132 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.850768 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81617) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 15:58:35.851660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81618) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 15:58:35.853653 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81617) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 15:58:35.855158 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81618) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 15:58:35.856598 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.858040 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.858150 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.859510 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81617) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 15:58:35.859596 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.859795 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.860223 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.861025 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81618) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 15:58:35.861312 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.861600 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.866530 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.867956 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81617) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 15:58:35.868190 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.869600 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81618) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 15:58:35.870836 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.871230 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.871517 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.871797 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.872500 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.872796 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.873086 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.873369 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.880038 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81617) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 15:58:35.880341 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.880622 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.881781 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81618) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 15:58:35.882084 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.882395 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.887421 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81617) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 15:58:35.887707 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.887992 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.888276 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.888552 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.888826 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.889103 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.889195 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81618) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 15:58:35.889393 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.889477 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.889666 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.889747 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.889950 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.890042 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.890222 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.890305 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.890536 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.890624 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.890812 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.890893 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.891095 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.891175 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81617) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.891393 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.892200 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.892478 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.892751 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.893035 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.893298 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81618) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 15:58:35.926697 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55a1f1d4a9c0 pid: 81617 (default app) Sep 19 15:58:35.928123 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55a1f1d4a9c0 pid: 81618 (default app) Sep 19 15:58:35.930005 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efebf65-faf7-4f04-aaa5-4b2760754f58 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:58:35.930287 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efebf65-faf7-4f04-aaa5-4b2760754f58 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:58:35.930583 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efebf65-faf7-4f04-aaa5-4b2760754f58 None None] PATH_INFO: `/v3` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:58:35.934699 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1/1] 10.222.0.57 () {50 vars in 791 bytes} [Mon Sep 19 15:58:35 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:58:42.835199 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddf1938-36d1-486f-8b44-8dccb48e91cf None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:58:42.835702 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddf1938-36d1-486f-8b44-8dccb48e91cf None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:58:42.835810 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddf1938-36d1-486f-8b44-8dccb48e91cf None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:58:42.840087 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1/2] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:58:42 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 15:58:42.845726 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa6a2c6-b7ab-4b81-b79e-c1b4ce0af4a9 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:58:42.846255 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa6a2c6-b7ab-4b81-b79e-c1b4ce0af4a9 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:58:42.846255 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa6a2c6-b7ab-4b81-b79e-c1b4ce0af4a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:58:42.925007 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: INFO dbcounter [None req-baa6a2c6-b7ab-4b81-b79e-c1b4ce0af4a9 None None] Registered counter for database keystone Sep 19 15:58:42.944569 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG oslo_db.sqlalchemy.engines [None req-baa6a2c6-b7ab-4b81-b79e-c1b4ce0af4a9 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81617) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 19 15:58:42.950699 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writer thread running {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 19 15:58:43.144240 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG passlib.handlers.bcrypt [None req-baa6a2c6-b7ab-4b81-b79e-c1b4ce0af4a9 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81617) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 19 15:58:43.144561 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG passlib.handlers.bcrypt [None req-baa6a2c6-b7ab-4b81-b79e-c1b4ce0af4a9 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81617) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 19 15:58:43.202886 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-baa6a2c6-b7ab-4b81-b79e-c1b4ce0af4a9 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:58:43.204540 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-baa6a2c6-b7ab-4b81-b79e-c1b4ce0af4a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:58:43.341101 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 2/3] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:58:42 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 496 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 19 15:58:43.797756 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddf1938-36d1-486f-8b44-8dccb48e91cf None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:58:43.867781 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: INFO dbcounter [None req-5ddf1938-36d1-486f-8b44-8dccb48e91cf None None] Registered counter for database keystone Sep 19 15:58:43.890489 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG oslo_db.sqlalchemy.engines [None req-5ddf1938-36d1-486f-8b44-8dccb48e91cf None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81618) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 19 15:58:43.898096 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writer thread running {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 19 15:58:44.021718 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddf1938-36d1-486f-8b44-8dccb48e91cf None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:58:44.026087 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605b2fcd-2a54-42da-a8e9-ef41500669f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:58:44.026760 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b2fcd-2a54-42da-a8e9-ef41500669f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:58:44.027038 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b2fcd-2a54-42da-a8e9-ef41500669f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:58:44.027354 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b2fcd-2a54-42da-a8e9-ef41500669f1 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:58:44.036688 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605b2fcd-2a54-42da-a8e9-ef41500669f1 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:58:44.088663 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: WARNING keystone.common.rbac_enforcer.enforcer [None req-605b2fcd-2a54-42da-a8e9-ef41500669f1 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 19 15:58:44.088995 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605b2fcd-2a54-42da-a8e9-ef41500669f1 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:58:44.099238 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 2/4] 10.222.0.57 () {58 vars in 1324 bytes} [Mon Sep 19 15:58:43 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 15:58:44.105264 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa6a2c6-b7ab-4b81-b79e-c1b4ce0af4a9 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:58:44.121985 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa6a2c6-b7ab-4b81-b79e-c1b4ce0af4a9 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:58:44.127242 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b904d-3ba1-48e0-ab53-32c66dcee19a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:58:44.128378 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b904d-3ba1-48e0-ab53-32c66dcee19a admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:58:44.128665 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b904d-3ba1-48e0-ab53-32c66dcee19a admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:58:44.128940 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b904d-3ba1-48e0-ab53-32c66dcee19a admin admin] PATH_INFO: `/v3/projects/5a804037a8734a35ad3ef5ec7e34d4c6` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:58:44.130597 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0b904d-3ba1-48e0-ab53-32c66dcee19a admin admin] RBAC: Authorizing `identity:get_project(project_id=5a804037a8734a35ad3ef5ec7e34d4c6)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:58:44.182894 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: WARNING keystone.common.rbac_enforcer.enforcer [None req-6e0b904d-3ba1-48e0-ab53-32c66dcee19a admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 19 15:58:44.183238 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0b904d-3ba1-48e0-ab53-32c66dcee19a admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:58:44.188516 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 3/5] 10.222.0.57 () {56 vars in 1207 bytes} [Mon Sep 19 15:58:44 2022] GET /identity/v3/projects/5a804037a8734a35ad3ef5ec7e34d4c6 => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 15:58:46.306305 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac217a7-e7b2-405b-b0f2-2bca2071c5a7 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:58:46.306886 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac217a7-e7b2-405b-b0f2-2bca2071c5a7 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:58:46.306886 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac217a7-e7b2-405b-b0f2-2bca2071c5a7 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:58:46.311014 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 3/6] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:58:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 15:58:46.317621 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e32369-eb70-4477-a207-c93cfd04fca7 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:58:46.318125 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e32369-eb70-4477-a207-c93cfd04fca7 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:58:46.318214 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e32369-eb70-4477-a207-c93cfd04fca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:58:46.339547 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-42e32369-eb70-4477-a207-c93cfd04fca7 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:58:46.341104 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-42e32369-eb70-4477-a207-c93cfd04fca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:58:46.349127 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 4/7] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:58:46 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 19 15:58:46.617943 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac217a7-e7b2-405b-b0f2-2bca2071c5a7 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:58:46.629960 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac217a7-e7b2-405b-b0f2-2bca2071c5a7 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:58:46.634139 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b672258c-a415-4934-b3f8-df1475f1d0fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:58:46.634765 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b672258c-a415-4934-b3f8-df1475f1d0fe admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:58:46.635047 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b672258c-a415-4934-b3f8-df1475f1d0fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:58:46.635321 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b672258c-a415-4934-b3f8-df1475f1d0fe admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:58:46.642846 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b672258c-a415-4934-b3f8-df1475f1d0fe admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:58:46.644595 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b672258c-a415-4934-b3f8-df1475f1d0fe admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:58:46.653133 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 4/8] 10.222.0.57 () {58 vars in 1324 bytes} [Mon Sep 19 15:58:46 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 15:58:46.660215 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e32369-eb70-4477-a207-c93cfd04fca7 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:58:46.670649 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e32369-eb70-4477-a207-c93cfd04fca7 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:58:46.675706 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839bd8fa-0a9a-47b1-9136-9357e4ac43f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:58:46.676334 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839bd8fa-0a9a-47b1-9136-9357e4ac43f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:58:46.676612 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839bd8fa-0a9a-47b1-9136-9357e4ac43f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:58:46.676883 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839bd8fa-0a9a-47b1-9136-9357e4ac43f5 admin admin] PATH_INFO: `/v3/users/18d4b3afb42f46999e6cf7b500999690` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:58:46.678189 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839bd8fa-0a9a-47b1-9136-9357e4ac43f5 admin admin] RBAC: Authorizing `identity:get_user(user_id=18d4b3afb42f46999e6cf7b500999690)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:58:46.680060 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839bd8fa-0a9a-47b1-9136-9357e4ac43f5 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:58:46.681597 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 5/9] 10.222.0.57 () {56 vars in 1198 bytes} [Mon Sep 19 15:58:46 2022] GET /identity/v3/users/18d4b3afb42f46999e6cf7b500999690 => generated 247 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:58:48.961537 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241a716f-ea8f-4e95-8173-a91b7423dbf9 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:58:48.962054 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241a716f-ea8f-4e95-8173-a91b7423dbf9 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:58:48.962162 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241a716f-ea8f-4e95-8173-a91b7423dbf9 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:58:48.963640 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 5/10] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:58:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 15:58:48.968673 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6724d820-8374-4672-a44b-a3b6c8211891 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:58:48.969138 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6724d820-8374-4672-a44b-a3b6c8211891 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:58:48.969883 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6724d820-8374-4672-a44b-a3b6c8211891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:58:48.987809 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-6724d820-8374-4672-a44b-a3b6c8211891 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:58:48.989009 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-6724d820-8374-4672-a44b-a3b6c8211891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:58:49.061106 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6decb553-fa2e-4932-ac8c-fee859d3f46d None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:58:49.061307 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6decb553-fa2e-4932-ac8c-fee859d3f46d None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:58:49.061582 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6decb553-fa2e-4932-ac8c-fee859d3f46d None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:58:49.062934 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 6/11] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:58:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 15:58:49.068141 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11db9b64-fb95-479b-87bc-47a51e446a7b None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:58:49.068871 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11db9b64-fb95-479b-87bc-47a51e446a7b None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:58:49.068871 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11db9b64-fb95-479b-87bc-47a51e446a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:58:49.071641 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 6/12] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:58:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 15:58:49.112154 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG passlib.handlers.bcrypt [None req-11db9b64-fb95-479b-87bc-47a51e446a7b None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81618) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 19 15:58:49.112541 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG passlib.handlers.bcrypt [None req-11db9b64-fb95-479b-87bc-47a51e446a7b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81618) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 19 15:58:49.162022 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-11db9b64-fb95-479b-87bc-47a51e446a7b None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:58:49.163707 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-11db9b64-fb95-479b-87bc-47a51e446a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:58:49.264807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 7/13] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:58:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 15:58:49.337238 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6724d820-8374-4672-a44b-a3b6c8211891 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:58:49.362512 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:58:49.447660 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6724d820-8374-4672-a44b-a3b6c8211891 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:58:49.481372 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30947340-e310-4b25-a351-69e7eabfdeab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:58:49.481983 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30947340-e310-4b25-a351-69e7eabfdeab None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:58:49.482262 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30947340-e310-4b25-a351-69e7eabfdeab None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:58:49.482566 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30947340-e310-4b25-a351-69e7eabfdeab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:58:49.528828 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11db9b64-fb95-479b-87bc-47a51e446a7b None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:58:49.549139 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30947340-e310-4b25-a351-69e7eabfdeab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:58:49.550647 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30947340-e310-4b25-a351-69e7eabfdeab None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:58:49.574991 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:58:49.630143 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11db9b64-fb95-479b-87bc-47a51e446a7b None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:58:49.632361 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:58:49.651520 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 7/14] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 19 15:58:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 19 15:58:49.657197 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30947340-e310-4b25-a351-69e7eabfdeab None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:58:49.668999 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619345f8-f3e1-44be-9cd1-82d5c8929f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:58:49.670107 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619345f8-f3e1-44be-9cd1-82d5c8929f4f None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:58:49.670414 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619345f8-f3e1-44be-9cd1-82d5c8929f4f None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:58:49.670692 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619345f8-f3e1-44be-9cd1-82d5c8929f4f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:58:49.682852 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619345f8-f3e1-44be-9cd1-82d5c8929f4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 15:58:49.691922 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-619345f8-f3e1-44be-9cd1-82d5c8929f4f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 15:58:49.691922 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 15:58:49.691922 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 15:58:49.691922 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 15:58:49.691922 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 15:58:49.691922 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 15:58:49.691922 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 15:58:49.691922 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 15:58:49.691922 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 15:58:49.691922 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 15:58:49.691922 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 15:58:49.691922 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 15:58:49.691922 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 15:58:49.691922 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 15:58:49.691922 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 15:58:49.691922 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 15:58:49.693126 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 15:58:49.693126 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 15:58:49.693126 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 15:58:49.693126 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 15:58:49.693126 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 15:58:49.693126 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 15:58:49.693126 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 15:58:49.693126 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 15:58:49.693126 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 15:58:49.693126 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 15:58:49.693126 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 15:58:49.693126 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 15:58:49.693126 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 15:58:49.693126 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 15:58:49.693126 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 15:58:49.693126 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 15:58:49.694145 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 15:58:49.694145 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 15:58:49.694145 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 15:58:49.694145 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 15:58:49.694145 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 15:58:49.694145 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 15:58:49.694145 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 15:58:49.694145 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 15:58:49.694145 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 15:58:49.694145 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 15:58:49.694145 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 15:58:49.694145 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 15:58:49.694145 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 8/15] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 19 15:58:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 15:58:49.698762 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619345f8-f3e1-44be-9cd1-82d5c8929f4f None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:58:49.765831 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30947340-e310-4b25-a351-69e7eabfdeab None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:58:49.790921 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:58:49.802605 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ffb15a-b09d-4df4-8374-d462cc3e6227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:58:49.803247 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ffb15a-b09d-4df4-8374-d462cc3e6227 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:58:49.803523 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ffb15a-b09d-4df4-8374-d462cc3e6227 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:58:49.803613 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619345f8-f3e1-44be-9cd1-82d5c8929f4f None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:58:49.803783 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ffb15a-b09d-4df4-8374-d462cc3e6227 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:58:49.814233 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ffb15a-b09d-4df4-8374-d462cc3e6227 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:58:49.815743 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ffb15a-b09d-4df4-8374-d462cc3e6227 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:58:49.835327 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-60ffb15a-b09d-4df4-8374-d462cc3e6227 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 15:58:49.835327 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 15:58:49.835327 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 15:58:49.835327 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 15:58:49.835327 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 15:58:49.835327 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 15:58:49.835327 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 15:58:49.835327 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 15:58:49.835327 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 15:58:49.835327 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 15:58:49.835327 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 15:58:49.835327 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 15:58:49.835327 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 15:58:49.835327 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 15:58:49.835327 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 15:58:49.835327 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 15:58:49.836304 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 15:58:49.836304 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 15:58:49.836304 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 15:58:49.836304 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 15:58:49.836304 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 15:58:49.836304 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 15:58:49.836304 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 15:58:49.836304 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 15:58:49.836304 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 15:58:49.836304 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 15:58:49.836304 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 15:58:49.836304 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 15:58:49.836304 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 15:58:49.836304 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 15:58:49.836304 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 15:58:49.836304 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 15:58:49.836304 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 15:58:49.837448 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 15:58:49.837448 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 15:58:49.837448 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 15:58:49.837448 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 8/16] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 15:58:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 15:58:49.839401 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5963f7-1dd9-44f7-afdd-0e43793eb0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:58:49.840083 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5963f7-1dd9-44f7-afdd-0e43793eb0f5 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:58:49.840368 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5963f7-1dd9-44f7-afdd-0e43793eb0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:58:49.840641 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5963f7-1dd9-44f7-afdd-0e43793eb0f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:58:49.841443 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5963f7-1dd9-44f7-afdd-0e43793eb0f5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:58:49.841750 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5963f7-1dd9-44f7-afdd-0e43793eb0f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:58:49.842257 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ffb15a-b09d-4df4-8374-d462cc3e6227 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:58:49.843215 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5963f7-1dd9-44f7-afdd-0e43793eb0f5 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:58:49.856366 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 9/17] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 15:58:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 15:58:49.862611 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5963f7-1dd9-44f7-afdd-0e43793eb0f5 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:58:49.890246 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:58:49.941982 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ffb15a-b09d-4df4-8374-d462cc3e6227 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:58:49.962696 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5963f7-1dd9-44f7-afdd-0e43793eb0f5 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:58:49.974961 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b45db72-38f9-4852-8a43-d14f2286b914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:58:49.975568 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b45db72-38f9-4852-8a43-d14f2286b914 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:58:49.975858 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b45db72-38f9-4852-8a43-d14f2286b914 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:58:49.976131 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b45db72-38f9-4852-8a43-d14f2286b914 None admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:58:49.976774 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b45db72-38f9-4852-8a43-d14f2286b914 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:58:49.977087 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b45db72-38f9-4852-8a43-d14f2286b914 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:58:49.978506 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b45db72-38f9-4852-8a43-d14f2286b914 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:58:49.995346 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 9/18] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 15:58:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 15:58:49.996511 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1275a0fb-1ebf-427e-998d-21103cf050b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:58:49.997146 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1275a0fb-1ebf-427e-998d-21103cf050b3 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:58:49.997425 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1275a0fb-1ebf-427e-998d-21103cf050b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:58:49.997699 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1275a0fb-1ebf-427e-998d-21103cf050b3 None admin] PATH_INFO: `/v3/users/18d4b3afb42f46999e6cf7b500999690` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:58:50.006048 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1275a0fb-1ebf-427e-998d-21103cf050b3 None admin] RBAC: Authorizing `identity:get_user(user_id=18d4b3afb42f46999e6cf7b500999690)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:58:50.006048 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1275a0fb-1ebf-427e-998d-21103cf050b3 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:58:50.006048 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 10/19] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 19 15:58:49 2022] GET /identity/v3/users/18d4b3afb42f46999e6cf7b500999690 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 15:58:50.008359 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b45db72-38f9-4852-8a43-d14f2286b914 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:58:50.046308 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:58:50.115888 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b45db72-38f9-4852-8a43-d14f2286b914 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:58:50.152387 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ab0389-266a-45ef-a826-d9c8f87d9194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:58:50.152946 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ab0389-266a-45ef-a826-d9c8f87d9194 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:58:50.153224 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ab0389-266a-45ef-a826-d9c8f87d9194 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:58:50.153492 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ab0389-266a-45ef-a826-d9c8f87d9194 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:58:50.154694 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ab0389-266a-45ef-a826-d9c8f87d9194 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:58:50.156164 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ab0389-266a-45ef-a826-d9c8f87d9194 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:58:50.157795 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 10/20] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 15:58:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:58:50.163698 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1275a0fb-1ebf-427e-998d-21103cf050b3 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:58:50.269737 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:58:50.276976 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1275a0fb-1ebf-427e-998d-21103cf050b3 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:58:50.314785 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898523ab-d31d-4f86-90a0-8e78f6e5cac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:58:50.315407 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898523ab-d31d-4f86-90a0-8e78f6e5cac5 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:58:50.315678 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898523ab-d31d-4f86-90a0-8e78f6e5cac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:58:50.315952 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898523ab-d31d-4f86-90a0-8e78f6e5cac5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:58:50.316680 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898523ab-d31d-4f86-90a0-8e78f6e5cac5 None admin] RBAC: Adding query filter params (role.id=63d2d197bfd6497583e94bfb065ce29b, scope.domain.id=default, user.id=18d4b3afb42f46999e6cf7b500999690) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:58:50.316987 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898523ab-d31d-4f86-90a0-8e78f6e5cac5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:58:50.318457 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898523ab-d31d-4f86-90a0-8e78f6e5cac5 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:58:50.327219 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 11/21] 10.222.0.57 () {56 vars in 1322 bytes} [Mon Sep 19 15:58:50 2022] GET /identity/v3/role_assignments?user.id=18d4b3afb42f46999e6cf7b500999690&scope.domain.id=default&role.id=63d2d197bfd6497583e94bfb065ce29b => generated 233 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:58:52.735513 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbff6f1-ba36-41bf-9a9e-c03763216332 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:58:52.736289 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbff6f1-ba36-41bf-9a9e-c03763216332 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:58:52.736289 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbff6f1-ba36-41bf-9a9e-c03763216332 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:58:52.737400 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 11/22] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:58:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 15:58:52.750931 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac0ef63-7844-49d0-9fb7-8e2d0665f23f None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:58:52.751447 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac0ef63-7844-49d0-9fb7-8e2d0665f23f None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:58:52.751447 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac0ef63-7844-49d0-9fb7-8e2d0665f23f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:58:52.780463 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-6ac0ef63-7844-49d0-9fb7-8e2d0665f23f None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:58:52.782242 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-6ac0ef63-7844-49d0-9fb7-8e2d0665f23f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:58:52.820589 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0abb51b-0399-4e18-abae-940ce6f81aa5 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:58:52.821255 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0abb51b-0399-4e18-abae-940ce6f81aa5 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:58:52.821735 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0abb51b-0399-4e18-abae-940ce6f81aa5 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:58:52.823537 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 12/23] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:58:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 15:58:52.830348 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d79988-9915-4983-b1f4-b3fb0a700747 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:58:52.830993 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d79988-9915-4983-b1f4-b3fb0a700747 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:58:52.831490 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d79988-9915-4983-b1f4-b3fb0a700747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:58:52.855506 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-98d79988-9915-4983-b1f4-b3fb0a700747 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:58:52.856747 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-98d79988-9915-4983-b1f4-b3fb0a700747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:58:52.900874 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 12/24] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:58:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 15:58:52.944285 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ad7231-a636-47d1-bfe8-ce7303cd2408 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:58:52.944922 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ad7231-a636-47d1-bfe8-ce7303cd2408 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:58:52.945404 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ad7231-a636-47d1-bfe8-ce7303cd2408 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:58:52.947014 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 13/25] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:58:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 15:58:52.952642 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7880451a-a2b7-445c-b5a4-638fbf647882 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:58:52.953296 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7880451a-a2b7-445c-b5a4-638fbf647882 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:58:52.953784 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7880451a-a2b7-445c-b5a4-638fbf647882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:58:52.978122 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-7880451a-a2b7-445c-b5a4-638fbf647882 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:58:52.980936 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-7880451a-a2b7-445c-b5a4-638fbf647882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:58:52.989028 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 13/26] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:58:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 15:58:53.123161 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 14/27] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:58:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 15:58:53.334455 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d79988-9915-4983-b1f4-b3fb0a700747 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:58:53.383488 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:58:53.401254 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a2247-cc8b-44c0-bbc7-27dc2cd0050e None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:58:53.401590 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a2247-cc8b-44c0-bbc7-27dc2cd0050e None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:58:53.401816 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a2247-cc8b-44c0-bbc7-27dc2cd0050e None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:58:53.403579 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 15/28] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:58:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 15:58:53.411080 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e804e7d-8230-4204-a6b3-24696d1889d3 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:58:53.411370 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e804e7d-8230-4204-a6b3-24696d1889d3 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:58:53.411632 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e804e7d-8230-4204-a6b3-24696d1889d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:58:53.455083 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-0e804e7d-8230-4204-a6b3-24696d1889d3 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:58:53.456280 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-0e804e7d-8230-4204-a6b3-24696d1889d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:58:53.498000 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:58:53.535925 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d79988-9915-4983-b1f4-b3fb0a700747 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:58:53.591563 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d123a533-46f6-46aa-bf7c-375ce6cf94fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:58:53.592100 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d123a533-46f6-46aa-bf7c-375ce6cf94fd None admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:58:53.592378 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d123a533-46f6-46aa-bf7c-375ce6cf94fd None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:58:53.592651 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d123a533-46f6-46aa-bf7c-375ce6cf94fd None admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:58:53.593346 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d123a533-46f6-46aa-bf7c-375ce6cf94fd None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:58:53.595323 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d123a533-46f6-46aa-bf7c-375ce6cf94fd None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:58:53.596249 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 16/29] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:58:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 15:58:53.597213 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e804e7d-8230-4204-a6b3-24696d1889d3 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:58:53.742565 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 14/30] 10.222.0.57 () {60 vars in 1131 bytes} [Mon Sep 19 15:58:53 2022] POST /identity/v3/roles => generated 219 bytes in 404 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 15:58:53.744532 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d123a533-46f6-46aa-bf7c-375ce6cf94fd None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:58:53.812282 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e804e7d-8230-4204-a6b3-24696d1889d3 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:58:53.892696 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2863a0-574f-4a48-aedb-39f4ecda1180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:58:53.893273 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2863a0-574f-4a48-aedb-39f4ecda1180 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:58:53.893556 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2863a0-574f-4a48-aedb-39f4ecda1180 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:58:53.894161 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2863a0-574f-4a48-aedb-39f4ecda1180 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:58:53.940283 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2863a0-574f-4a48-aedb-39f4ecda1180 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:58:53.941803 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2863a0-574f-4a48-aedb-39f4ecda1180 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:58:53.954759 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:58:53.955796 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:58:53.965572 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d123a533-46f6-46aa-bf7c-375ce6cf94fd None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:58:53.971062 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-2b2863a0-574f-4a48-aedb-39f4ecda1180 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 15:58:53.971062 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 15:58:53.971062 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 15:58:53.971062 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 15:58:53.971062 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 15:58:53.971062 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 15:58:53.971062 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 15:58:53.971062 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 15:58:53.971062 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 15:58:53.971062 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 15:58:53.971062 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 15:58:53.971062 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 15:58:53.971062 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 15:58:53.971062 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 15:58:53.971062 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 15:58:53.971062 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 15:58:53.971897 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 15:58:53.971897 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 15:58:53.971897 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 15:58:53.971897 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 15:58:53.971897 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 15:58:53.971897 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 15:58:53.971897 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 15:58:53.971897 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 15:58:53.971897 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 15:58:53.971897 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 15:58:53.971897 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 15:58:53.971897 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 15:58:53.971897 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 15:58:53.971897 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 15:58:53.971897 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 15:58:53.971897 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 15:58:53.971897 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 15:58:53.972679 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 15:58:53.972679 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 15:58:53.972679 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 15:58:53.984141 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 17/31] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 15:58:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 387 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 15:58:53.985874 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2863a0-574f-4a48-aedb-39f4ecda1180 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:58:54.010039 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864c35a7-729d-4b88-89e3-7f44454266eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:58:54.010710 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c35a7-729d-4b88-89e3-7f44454266eb None admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:58:54.010999 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c35a7-729d-4b88-89e3-7f44454266eb None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:58:54.011273 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c35a7-729d-4b88-89e3-7f44454266eb None admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:58:54.011969 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864c35a7-729d-4b88-89e3-7f44454266eb None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:58:54.018401 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864c35a7-729d-4b88-89e3-7f44454266eb None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:58:54.037105 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 15/32] 10.222.0.57 () {60 vars in 1131 bytes} [Mon Sep 19 15:58:53 2022] POST /identity/v3/roles => generated 225 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 15:58:54.039274 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864c35a7-729d-4b88-89e3-7f44454266eb None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:58:54.223477 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2863a0-574f-4a48-aedb-39f4ecda1180 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:58:54.237692 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864c35a7-729d-4b88-89e3-7f44454266eb None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:58:54.294169 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586888dd-fd1c-451c-8ccf-400d5d575ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:58:54.294824 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586888dd-fd1c-451c-8ccf-400d5d575ec9 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:58:54.295111 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586888dd-fd1c-451c-8ccf-400d5d575ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:58:54.295393 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586888dd-fd1c-451c-8ccf-400d5d575ec9 None admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:58:54.296059 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586888dd-fd1c-451c-8ccf-400d5d575ec9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:58:54.296369 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586888dd-fd1c-451c-8ccf-400d5d575ec9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:58:54.297890 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586888dd-fd1c-451c-8ccf-400d5d575ec9 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:58:54.300720 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9721583d-f05a-4d09-8c7e-fb37f56b17e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:58:54.302037 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9721583d-f05a-4d09-8c7e-fb37f56b17e1 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:58:54.302318 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9721583d-f05a-4d09-8c7e-fb37f56b17e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:58:54.302637 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9721583d-f05a-4d09-8c7e-fb37f56b17e1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:58:54.314070 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9721583d-f05a-4d09-8c7e-fb37f56b17e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 15:58:54.318108 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 16/33] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 15:58:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:58:54.318108 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445e9be-2fd8-4a8c-adf3-a48a71b7cbb8 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:58:54.318108 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445e9be-2fd8-4a8c-adf3-a48a71b7cbb8 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:58:54.318108 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445e9be-2fd8-4a8c-adf3-a48a71b7cbb8 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:58:54.318108 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 17/34] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:58:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 15:58:54.319693 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-9721583d-f05a-4d09-8c7e-fb37f56b17e1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 15:58:54.319693 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 15:58:54.319693 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 15:58:54.319693 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 15:58:54.319693 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 15:58:54.319693 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 15:58:54.319693 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 15:58:54.319693 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 15:58:54.319693 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 15:58:54.319693 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 15:58:54.319693 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 15:58:54.319693 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 15:58:54.319693 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 15:58:54.319693 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 15:58:54.319693 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 15:58:54.319693 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 15:58:54.320704 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 15:58:54.320704 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 15:58:54.320704 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 15:58:54.320704 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 15:58:54.320704 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 15:58:54.320704 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 15:58:54.320704 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 15:58:54.320704 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 15:58:54.320704 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 15:58:54.320704 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 15:58:54.320704 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 15:58:54.320704 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 15:58:54.320704 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 15:58:54.320704 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 15:58:54.320704 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 15:58:54.320704 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 15:58:54.321593 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 15:58:54.321593 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 15:58:54.321593 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 15:58:54.321593 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 15:58:54.321593 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 15:58:54.321593 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 15:58:54.321593 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 15:58:54.321593 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 15:58:54.321593 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 15:58:54.321593 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 15:58:54.321593 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 15:58:54.321593 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 15:58:54.321593 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 18/35] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 19 15:58:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 15:58:54.326131 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d445e9be-2fd8-4a8c-adf3-a48a71b7cbb8 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:58:54.327628 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c602718-11a8-4f2a-aab8-db4553119128 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:58:54.327907 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c602718-11a8-4f2a-aab8-db4553119128 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:58:54.328703 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c602718-11a8-4f2a-aab8-db4553119128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:58:54.349182 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-0c602718-11a8-4f2a-aab8-db4553119128 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:58:54.350407 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-0c602718-11a8-4f2a-aab8-db4553119128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:58:54.351709 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:58:54.414909 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:58:54.426422 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 19/36] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:58:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 15:58:54.431538 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c602718-11a8-4f2a-aab8-db4553119128 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:58:54.553649 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c602718-11a8-4f2a-aab8-db4553119128 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:58:54.560443 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d445e9be-2fd8-4a8c-adf3-a48a71b7cbb8 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:58:54.601558 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0956e457-2d00-4d6c-92a3-678452211361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:58:54.602172 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0956e457-2d00-4d6c-92a3-678452211361 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:58:54.602487 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0956e457-2d00-4d6c-92a3-678452211361 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:58:54.602765 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0956e457-2d00-4d6c-92a3-678452211361 None admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:58:54.603436 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0956e457-2d00-4d6c-92a3-678452211361 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:58:54.603763 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0956e457-2d00-4d6c-92a3-678452211361 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:58:54.605111 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0956e457-2d00-4d6c-92a3-678452211361 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:58:54.605435 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e73d090-a964-40f7-a7ed-a118aa289932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:58:54.606259 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e73d090-a964-40f7-a7ed-a118aa289932 None admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:58:54.606782 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e73d090-a964-40f7-a7ed-a118aa289932 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:58:54.607250 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e73d090-a964-40f7-a7ed-a118aa289932 None admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:58:54.608117 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e73d090-a964-40f7-a7ed-a118aa289932 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:58:54.609894 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e73d090-a964-40f7-a7ed-a118aa289932 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:58:54.613923 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 20/37] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 15:58:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:58:54.619718 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0956e457-2d00-4d6c-92a3-678452211361 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:58:54.648703 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 18/38] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Sep 19 15:58:54 2022] POST /identity/v3/projects => generated 301 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 15:58:54.688706 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d72101-c3b5-4eb2-928b-eb1f2ac131c2 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:58:54.689013 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d72101-c3b5-4eb2-928b-eb1f2ac131c2 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:58:54.689264 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d72101-c3b5-4eb2-928b-eb1f2ac131c2 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:58:54.690882 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 19/39] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:58:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 15:58:54.696690 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84bf8a4-1e4e-49fe-ba50-4a33af1fef3d None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:58:54.696992 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84bf8a4-1e4e-49fe-ba50-4a33af1fef3d None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:58:54.697245 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84bf8a4-1e4e-49fe-ba50-4a33af1fef3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:58:54.709076 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:58:54.721083 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-e84bf8a4-1e4e-49fe-ba50-4a33af1fef3d None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:58:54.723007 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-e84bf8a4-1e4e-49fe-ba50-4a33af1fef3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:58:54.766958 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0956e457-2d00-4d6c-92a3-678452211361 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:58:54.796213 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:58:54.807002 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec11f038-20e5-4485-aa84-d1604357396e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:58:54.807651 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 20/40] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:58:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 15:58:54.807756 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec11f038-20e5-4485-aa84-d1604357396e None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:58:54.807925 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec11f038-20e5-4485-aa84-d1604357396e None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:58:54.808194 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec11f038-20e5-4485-aa84-d1604357396e None admin] PATH_INFO: `/v3/users/18d4b3afb42f46999e6cf7b500999690` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:58:54.809577 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec11f038-20e5-4485-aa84-d1604357396e None admin] RBAC: Authorizing `identity:get_user(user_id=18d4b3afb42f46999e6cf7b500999690)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:58:54.811083 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec11f038-20e5-4485-aa84-d1604357396e None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:58:54.818182 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 21/41] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 19 15:58:54 2022] GET /identity/v3/users/18d4b3afb42f46999e6cf7b500999690 => generated 247 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 15:58:54.818989 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84bf8a4-1e4e-49fe-ba50-4a33af1fef3d None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:58:54.819185 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec11f038-20e5-4485-aa84-d1604357396e None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:58:54.944627 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84bf8a4-1e4e-49fe-ba50-4a33af1fef3d None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:58:54.959541 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec11f038-20e5-4485-aa84-d1604357396e None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:58:54.981852 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f7947-2fc9-40ae-b342-4f7fddd2e147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:58:54.982503 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f7947-2fc9-40ae-b342-4f7fddd2e147 None admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:58:54.982821 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f7947-2fc9-40ae-b342-4f7fddd2e147 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:58:54.983104 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f7947-2fc9-40ae-b342-4f7fddd2e147 None admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:58:54.983778 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73f7947-2fc9-40ae-b342-4f7fddd2e147 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:58:54.985215 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73f7947-2fc9-40ae-b342-4f7fddd2e147 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:58:54.986985 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:58:55.000008 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 21/42] 10.222.0.57 () {60 vars in 1131 bytes} [Mon Sep 19 15:58:54 2022] POST /identity/v3/roles => generated 223 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 15:58:55.008102 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac9a980-8c23-4f42-9151-689392a64626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:58:55.008764 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9a980-8c23-4f42-9151-689392a64626 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:58:55.009095 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9a980-8c23-4f42-9151-689392a64626 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:58:55.009375 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9a980-8c23-4f42-9151-689392a64626 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:58:55.010717 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac9a980-8c23-4f42-9151-689392a64626 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:58:55.012210 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac9a980-8c23-4f42-9151-689392a64626 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:58:55.018723 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 22/43] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 15:58:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:58:55.021269 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f7947-2fc9-40ae-b342-4f7fddd2e147 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:58:55.083903 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac9a980-8c23-4f42-9151-689392a64626 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:58:55.088107 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:58:55.144344 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f7947-2fc9-40ae-b342-4f7fddd2e147 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:58:55.184232 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b397c0a-018f-4d9c-a809-a6e31b28e870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:58:55.184906 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b397c0a-018f-4d9c-a809-a6e31b28e870 None admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:58:55.185183 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b397c0a-018f-4d9c-a809-a6e31b28e870 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:58:55.185458 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b397c0a-018f-4d9c-a809-a6e31b28e870 None admin] PATH_INFO: `/v3/domains/default/users/18d4b3afb42f46999e6cf7b500999690/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:58:55.188434 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b397c0a-018f-4d9c-a809-a6e31b28e870 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=18d4b3afb42f46999e6cf7b500999690, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:58:55.188990 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b397c0a-018f-4d9c-a809-a6e31b28e870 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:58:55.205828 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 22/44] 10.222.0.57 () {58 vars in 1361 bytes} [Mon Sep 19 15:58:55 2022] PUT /identity/v3/domains/default/users/18d4b3afb42f46999e6cf7b500999690/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:58:55.206254 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac9a980-8c23-4f42-9151-689392a64626 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:58:55.247575 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46debed5-236c-49fe-8b17-520363f460ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:58:55.248465 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46debed5-236c-49fe-8b17-520363f460ed None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:58:55.248883 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46debed5-236c-49fe-8b17-520363f460ed None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:58:55.249292 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46debed5-236c-49fe-8b17-520363f460ed None admin] PATH_INFO: `/v3/domains/default` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:58:55.251189 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46debed5-236c-49fe-8b17-520363f460ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:58:55.252820 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46debed5-236c-49fe-8b17-520363f460ed None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:58:55.254452 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 23/45] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 15:58:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 15:58:55.260276 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b397c0a-018f-4d9c-a809-a6e31b28e870 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:58:55.368004 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b397c0a-018f-4d9c-a809-a6e31b28e870 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:58:55.384005 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:58:55.405436 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d92744-b1cb-49b7-a12c-131d00ce1f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:58:55.406057 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d92744-b1cb-49b7-a12c-131d00ce1f7c None admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:58:55.406883 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d92744-b1cb-49b7-a12c-131d00ce1f7c None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:58:55.407164 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d92744-b1cb-49b7-a12c-131d00ce1f7c None admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:58:55.407841 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d92744-b1cb-49b7-a12c-131d00ce1f7c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:58:55.409297 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d92744-b1cb-49b7-a12c-131d00ce1f7c None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:58:55.439347 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 23/46] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Sep 19 15:58:55 2022] POST /identity/v3/projects => generated 312 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 15:58:57.372224 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762f5c79-151d-4670-a182-f81ed191da78 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:58:57.372738 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762f5c79-151d-4670-a182-f81ed191da78 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:58:57.372738 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762f5c79-151d-4670-a182-f81ed191da78 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:58:57.374077 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 24/47] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:58:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 15:58:57.379911 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781a7834-13f5-47e9-b084-7ab316f0a8e0 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:58:57.380298 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781a7834-13f5-47e9-b084-7ab316f0a8e0 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:58:57.380561 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781a7834-13f5-47e9-b084-7ab316f0a8e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:58:57.398892 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-781a7834-13f5-47e9-b084-7ab316f0a8e0 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:58:57.400110 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-781a7834-13f5-47e9-b084-7ab316f0a8e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:58:57.465845 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 24/48] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:58:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 15:58:57.675823 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a12176-5611-4d83-bfa3-3a33a78e5b18 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:58:57.676069 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a12176-5611-4d83-bfa3-3a33a78e5b18 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:58:57.676303 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a12176-5611-4d83-bfa3-3a33a78e5b18 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:58:57.677642 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 25/49] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:58:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 15:58:57.683058 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc63da-1311-4791-91c5-8314008a0b2d None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:58:57.683351 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc63da-1311-4791-91c5-8314008a0b2d None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:58:57.683632 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc63da-1311-4791-91c5-8314008a0b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:58:57.701944 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-1fcc63da-1311-4791-91c5-8314008a0b2d None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:58:57.703865 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-1fcc63da-1311-4791-91c5-8314008a0b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:58:57.742048 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a12176-5611-4d83-bfa3-3a33a78e5b18 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:58:57.775715 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 25/50] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:58:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 15:58:57.809756 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:58:57.855382 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a12176-5611-4d83-bfa3-3a33a78e5b18 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:58:57.892525 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6469057f-11c6-4f99-9124-f8a0b6b241ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:58:57.893155 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469057f-11c6-4f99-9124-f8a0b6b241ff None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:58:57.893433 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469057f-11c6-4f99-9124-f8a0b6b241ff None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:58:57.893706 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469057f-11c6-4f99-9124-f8a0b6b241ff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:58:57.904892 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6469057f-11c6-4f99-9124-f8a0b6b241ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 15:58:57.909776 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-6469057f-11c6-4f99-9124-f8a0b6b241ff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 15:58:57.909776 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 15:58:57.909776 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 15:58:57.909776 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 15:58:57.909776 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 15:58:57.909776 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 15:58:57.909776 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 15:58:57.909776 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 15:58:57.909776 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 15:58:57.909776 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 15:58:57.909776 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 15:58:57.909776 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 15:58:57.909776 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 15:58:57.909776 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 15:58:57.909776 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 15:58:57.909776 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 15:58:57.910589 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 15:58:57.910589 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 15:58:57.910589 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 15:58:57.910589 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 15:58:57.910589 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 15:58:57.910589 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 15:58:57.910589 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 15:58:57.910589 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 15:58:57.910589 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 15:58:57.910589 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 15:58:57.910589 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 15:58:57.910589 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 15:58:57.910589 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 15:58:57.910589 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 15:58:57.910589 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 15:58:57.910589 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 15:58:57.911348 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 15:58:57.911348 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 15:58:57.911348 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 15:58:57.911348 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 15:58:57.911348 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 15:58:57.911348 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 15:58:57.911348 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 15:58:57.911348 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 15:58:57.911348 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 15:58:57.911348 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 15:58:57.911348 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 15:58:57.911348 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 15:58:57.914531 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 26/51] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 19 15:58:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 15:58:57.916962 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcc63da-1311-4791-91c5-8314008a0b2d None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:58:57.945080 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:58:58.020723 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcc63da-1311-4791-91c5-8314008a0b2d None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:58:58.047264 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6469057f-11c6-4f99-9124-f8a0b6b241ff None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:58:58.056073 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8728df-5162-4d27-ab57-8f5fcb498ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:58:58.056695 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8728df-5162-4d27-ab57-8f5fcb498ef0 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:58:58.056980 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8728df-5162-4d27-ab57-8f5fcb498ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:58:58.057254 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8728df-5162-4d27-ab57-8f5fcb498ef0 None admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:58:58.057928 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8728df-5162-4d27-ab57-8f5fcb498ef0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:58:58.058230 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8728df-5162-4d27-ab57-8f5fcb498ef0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:58:58.059671 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8728df-5162-4d27-ab57-8f5fcb498ef0 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:58:58.072529 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 26/52] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 15:58:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:58:58.078583 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8728df-5162-4d27-ab57-8f5fcb498ef0 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:58:58.153811 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6469057f-11c6-4f99-9124-f8a0b6b241ff None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:58:58.172717 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:58:58.174416 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:58:58.188155 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8728df-5162-4d27-ab57-8f5fcb498ef0 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:58:58.194110 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0692d2d5-785d-46e6-93ad-23592fe3a329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:58:58.194812 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0692d2d5-785d-46e6-93ad-23592fe3a329 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:58:58.195142 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0692d2d5-785d-46e6-93ad-23592fe3a329 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:58:58.195362 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0692d2d5-785d-46e6-93ad-23592fe3a329 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:58:58.196498 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0692d2d5-785d-46e6-93ad-23592fe3a329 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:58:58.197968 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0692d2d5-785d-46e6-93ad-23592fe3a329 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:58:58.199629 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 27/53] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 15:58:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 15:58:58.205608 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0692d2d5-785d-46e6-93ad-23592fe3a329 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:58:58.224251 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84a420c-abcc-4af8-9602-73a34c5e0164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:58:58.224907 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84a420c-abcc-4af8-9602-73a34c5e0164 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:58:58.225190 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84a420c-abcc-4af8-9602-73a34c5e0164 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:58:58.225463 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84a420c-abcc-4af8-9602-73a34c5e0164 None admin] PATH_INFO: `/v3/users/18d4b3afb42f46999e6cf7b500999690` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:58:58.226744 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84a420c-abcc-4af8-9602-73a34c5e0164 None admin] RBAC: Authorizing `identity:get_user(user_id=18d4b3afb42f46999e6cf7b500999690)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:58:58.228248 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84a420c-abcc-4af8-9602-73a34c5e0164 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:58:58.229912 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 27/54] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 19 15:58:58 2022] GET /identity/v3/users/18d4b3afb42f46999e6cf7b500999690 => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 15:58:58.236392 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84a420c-abcc-4af8-9602-73a34c5e0164 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:58:58.314689 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0692d2d5-785d-46e6-93ad-23592fe3a329 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:58:58.348062 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84a420c-abcc-4af8-9602-73a34c5e0164 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:58:58.350286 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671cc8db-65bf-413b-9899-daa500812a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:58:58.350944 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671cc8db-65bf-413b-9899-daa500812a26 None admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:58:58.351228 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671cc8db-65bf-413b-9899-daa500812a26 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:58:58.351505 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671cc8db-65bf-413b-9899-daa500812a26 None admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:58:58.352159 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671cc8db-65bf-413b-9899-daa500812a26 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:58:58.353607 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671cc8db-65bf-413b-9899-daa500812a26 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:58:58.381928 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989bb0ce-1489-4795-b032-e3003afc9941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:58:58.383119 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989bb0ce-1489-4795-b032-e3003afc9941 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:58:58.383407 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989bb0ce-1489-4795-b032-e3003afc9941 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:58:58.383686 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989bb0ce-1489-4795-b032-e3003afc9941 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:58:58.384839 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989bb0ce-1489-4795-b032-e3003afc9941 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:58:58.386280 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989bb0ce-1489-4795-b032-e3003afc9941 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:58:58.387892 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 28/55] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 15:58:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 15:58:58.393720 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989bb0ce-1489-4795-b032-e3003afc9941 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:58:58.419775 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:58:58.496699 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989bb0ce-1489-4795-b032-e3003afc9941 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:58:58.534604 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a80fa1-0a32-4c08-bc0a-3ddbb8f7f736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:58:58.535238 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a80fa1-0a32-4c08-bc0a-3ddbb8f7f736 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:58:58.535518 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a80fa1-0a32-4c08-bc0a-3ddbb8f7f736 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:58:58.535792 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a80fa1-0a32-4c08-bc0a-3ddbb8f7f736 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:58:58.536756 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a80fa1-0a32-4c08-bc0a-3ddbb8f7f736 None admin] RBAC: Adding query filter params (role.id=63d2d197bfd6497583e94bfb065ce29b, scope.domain.id=default, user.id=18d4b3afb42f46999e6cf7b500999690) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:58:58.537068 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a80fa1-0a32-4c08-bc0a-3ddbb8f7f736 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:58:58.538593 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a80fa1-0a32-4c08-bc0a-3ddbb8f7f736 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:58:58.547590 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 29/56] 10.222.0.57 () {56 vars in 1322 bytes} [Mon Sep 19 15:58:58 2022] GET /identity/v3/role_assignments?user.id=18d4b3afb42f46999e6cf7b500999690&scope.domain.id=default&role.id=63d2d197bfd6497583e94bfb065ce29b => generated 530 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 15:58:58.597011 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 28/57] 10.222.0.57 () {60 vars in 1140 bytes} [Mon Sep 19 15:58:58 2022] POST /identity/v3/projects => generated 298 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 15:59:00.779092 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee280e6-200f-46e6-a6f8-d7380e7c485d None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:00.779465 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee280e6-200f-46e6-a6f8-d7380e7c485d None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:00.779652 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee280e6-200f-46e6-a6f8-d7380e7c485d None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:00.781033 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 30/58] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:59:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 15:59:00.786708 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0ca53d-9ca9-4eda-9f0c-025db80ecd0b None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:00.787140 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0ca53d-9ca9-4eda-9f0c-025db80ecd0b None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:00.787272 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0ca53d-9ca9-4eda-9f0c-025db80ecd0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:00.806056 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-2c0ca53d-9ca9-4eda-9f0c-025db80ecd0b None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:59:00.807597 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-2c0ca53d-9ca9-4eda-9f0c-025db80ecd0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:59:00.836191 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:00.872805 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 29/59] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:59:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 15:59:01.151632 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee280e6-200f-46e6-a6f8-d7380e7c485d None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:01.244309 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:01.257882 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee280e6-200f-46e6-a6f8-d7380e7c485d None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:01.292266 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1e42b0-857f-47f4-8504-d155d5ba33c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:01.292894 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e42b0-857f-47f4-8504-d155d5ba33c0 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:01.293176 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e42b0-857f-47f4-8504-d155d5ba33c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:01.293451 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e42b0-857f-47f4-8504-d155d5ba33c0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:01.294594 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1e42b0-857f-47f4-8504-d155d5ba33c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:01.296053 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1e42b0-857f-47f4-8504-d155d5ba33c0 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:01.298263 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 31/60] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 15:59:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:01.303775 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0ca53d-9ca9-4eda-9f0c-025db80ecd0b None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:01.398854 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0ca53d-9ca9-4eda-9f0c-025db80ecd0b None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:01.431297 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016672b8-5fad-4afd-81fa-732aff158551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:01.431826 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016672b8-5fad-4afd-81fa-732aff158551 None admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:01.432106 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016672b8-5fad-4afd-81fa-732aff158551 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:01.432380 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016672b8-5fad-4afd-81fa-732aff158551 None admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:01.433039 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016672b8-5fad-4afd-81fa-732aff158551 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:01.434644 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016672b8-5fad-4afd-81fa-732aff158551 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:01.461783 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 30/61] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Sep 19 15:59:01 2022] POST /identity/v3/users => generated 275 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 15:59:04.576023 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054f6e2f-231a-446a-b593-70173c002acd None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:04.577038 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054f6e2f-231a-446a-b593-70173c002acd None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:04.577555 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054f6e2f-231a-446a-b593-70173c002acd None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:04.579749 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 32/62] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:59:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 15:59:04.585374 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c419a49-9fa4-456f-91ea-cef0c2842b9f None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:04.586221 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c419a49-9fa4-456f-91ea-cef0c2842b9f None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:04.586751 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c419a49-9fa4-456f-91ea-cef0c2842b9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:04.611463 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-4c419a49-9fa4-456f-91ea-cef0c2842b9f None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:59:04.612680 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-4c419a49-9fa4-456f-91ea-cef0c2842b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:59:04.634646 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:04.700747 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 31/63] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:59:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 15:59:05.023759 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054f6e2f-231a-446a-b593-70173c002acd None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:05.044273 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f62e200-9ce1-4040-b02d-59ee667ff5db None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:05.044945 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f62e200-9ce1-4040-b02d-59ee667ff5db None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:05.045433 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f62e200-9ce1-4040-b02d-59ee667ff5db None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:05.047078 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 32/64] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:59:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 15:59:05.055896 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15352ce0-d593-459e-b0e3-acc08036e310 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:05.056505 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15352ce0-d593-459e-b0e3-acc08036e310 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:05.057211 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15352ce0-d593-459e-b0e3-acc08036e310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:05.090468 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-15352ce0-d593-459e-b0e3-acc08036e310 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:59:05.092034 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-15352ce0-d593-459e-b0e3-acc08036e310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:59:05.220097 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054f6e2f-231a-446a-b593-70173c002acd None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:05.240072 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 33/65] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:59:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 15:59:05.285235 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d83a56-d9a1-47e9-9317-ed25fad926b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:05.286232 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d83a56-d9a1-47e9-9317-ed25fad926b0 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:05.290952 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d83a56-d9a1-47e9-9317-ed25fad926b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:05.291803 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d83a56-d9a1-47e9-9317-ed25fad926b0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:05.307419 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d83a56-d9a1-47e9-9317-ed25fad926b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 15:59:05.320101 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-47d83a56-d9a1-47e9-9317-ed25fad926b0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 15:59:05.320101 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 15:59:05.320101 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 15:59:05.320101 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 15:59:05.320101 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 15:59:05.320101 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 15:59:05.320101 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 15:59:05.320101 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 15:59:05.320101 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 15:59:05.320101 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 15:59:05.320101 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 15:59:05.320101 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 15:59:05.320101 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 15:59:05.320101 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 15:59:05.320101 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 15:59:05.320101 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 15:59:05.321087 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 15:59:05.321087 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 15:59:05.321087 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 15:59:05.321087 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 15:59:05.321087 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 15:59:05.321087 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 15:59:05.321087 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 15:59:05.321087 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 15:59:05.321087 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 15:59:05.321087 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 15:59:05.321087 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 15:59:05.321087 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 15:59:05.321087 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 15:59:05.321087 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 15:59:05.321087 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 15:59:05.321087 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 15:59:05.321826 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 15:59:05.321826 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 15:59:05.321826 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 15:59:05.321826 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 15:59:05.321826 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 15:59:05.321826 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 15:59:05.321826 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 15:59:05.321826 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 15:59:05.321826 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 15:59:05.321826 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 15:59:05.321826 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 15:59:05.321826 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 15:59:05.324252 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 33/66] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 15:59:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 15:59:05.330942 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15352ce0-d593-459e-b0e3-acc08036e310 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:05.357580 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7072150-f2ed-4d81-b120-d6506d453e02 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:05.358191 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7072150-f2ed-4d81-b120-d6506d453e02 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:05.358691 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7072150-f2ed-4d81-b120-d6506d453e02 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:05.360355 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 34/67] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:59:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 15:59:05.366037 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b19e9c-9b0d-42c4-9fcb-7b6c3b11bbd6 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:05.366677 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b19e9c-9b0d-42c4-9fcb-7b6c3b11bbd6 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:05.367146 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b19e9c-9b0d-42c4-9fcb-7b6c3b11bbd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:05.400251 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-31b19e9c-9b0d-42c4-9fcb-7b6c3b11bbd6 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:59:05.401527 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-31b19e9c-9b0d-42c4-9fcb-7b6c3b11bbd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:59:05.426074 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:05.468762 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:05.487044 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15352ce0-d593-459e-b0e3-acc08036e310 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:05.509094 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 35/68] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:59:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 15:59:05.511614 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2fb949-e5ac-4e7a-983f-2d6efad7e492 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:05.512157 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2fb949-e5ac-4e7a-983f-2d6efad7e492 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:05.512765 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2fb949-e5ac-4e7a-983f-2d6efad7e492 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:05.514883 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 36/69] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:59:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 15:59:05.521352 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c8bc99-bf1d-4ec4-b9b9-d9ef3a5cdd8a None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:05.522636 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c8bc99-bf1d-4ec4-b9b9-d9ef3a5cdd8a None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:05.523120 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c8bc99-bf1d-4ec4-b9b9-d9ef3a5cdd8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:05.531663 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d394746-c9cb-4a91-8351-46b5d8fb3c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:05.532662 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d394746-c9cb-4a91-8351-46b5d8fb3c7f None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:05.533120 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d394746-c9cb-4a91-8351-46b5d8fb3c7f None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:05.533570 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d394746-c9cb-4a91-8351-46b5d8fb3c7f None admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:05.534512 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d394746-c9cb-4a91-8351-46b5d8fb3c7f None admin] RBAC: Adding query filter params (name=member) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:05.534998 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d394746-c9cb-4a91-8351-46b5d8fb3c7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:05.536679 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d394746-c9cb-4a91-8351-46b5d8fb3c7f None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:05.544955 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-65c8bc99-bf1d-4ec4-b9b9-d9ef3a5cdd8a None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:59:05.546295 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-65c8bc99-bf1d-4ec4-b9b9-d9ef3a5cdd8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:59:05.547134 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 34/70] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Sep 19 15:59:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:05.548812 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb678f3-e28a-47b5-a4e3-1419b46d2f06 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:05.549498 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb678f3-e28a-47b5-a4e3-1419b46d2f06 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:05.549978 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb678f3-e28a-47b5-a4e3-1419b46d2f06 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:05.552558 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 35/71] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:59:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 15:59:05.553504 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb678f3-e28a-47b5-a4e3-1419b46d2f06 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:05.627123 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 37/72] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:59:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 15:59:05.636285 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f311c5a8-bf4c-46b0-bf0f-c87918278d01 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:05.636692 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f311c5a8-bf4c-46b0-bf0f-c87918278d01 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:05.637078 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f311c5a8-bf4c-46b0-bf0f-c87918278d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:05.662262 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-f311c5a8-bf4c-46b0-bf0f-c87918278d01 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:59:05.663392 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb678f3-e28a-47b5-a4e3-1419b46d2f06 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:05.663816 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-f311c5a8-bf4c-46b0-bf0f-c87918278d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:59:05.704964 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3075d0b-5b8e-4fa3-90ef-732f968a8302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:05.705605 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3075d0b-5b8e-4fa3-90ef-732f968a8302 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:05.705890 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3075d0b-5b8e-4fa3-90ef-732f968a8302 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:05.706167 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3075d0b-5b8e-4fa3-90ef-732f968a8302 None admin] PATH_INFO: `/v3/users/bbf2a95469bc4f3593466124dfc85117` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:05.725288 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3075d0b-5b8e-4fa3-90ef-732f968a8302 None admin] RBAC: Authorizing `identity:get_user(user_id=bbf2a95469bc4f3593466124dfc85117)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:05.727455 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3075d0b-5b8e-4fa3-90ef-732f968a8302 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:05.729700 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 36/73] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 19 15:59:05 2022] GET /identity/v3/users/bbf2a95469bc4f3593466124dfc85117 => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:05.731076 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3075d0b-5b8e-4fa3-90ef-732f968a8302 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:05.736999 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:05.773317 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 38/74] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:59:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 15:59:05.774625 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f311c5a8-bf4c-46b0-bf0f-c87918278d01 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:05.786507 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:05.869855 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3075d0b-5b8e-4fa3-90ef-732f968a8302 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:05.887253 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f311c5a8-bf4c-46b0-bf0f-c87918278d01 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:05.916612 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309fbf1b-6301-4fb9-88c3-697984ee270c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:05.917406 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309fbf1b-6301-4fb9-88c3-697984ee270c None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:05.917690 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309fbf1b-6301-4fb9-88c3-697984ee270c None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:05.917967 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309fbf1b-6301-4fb9-88c3-697984ee270c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:05.923881 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e671f43-db24-47bb-9787-20d9bf00490d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:05.924495 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e671f43-db24-47bb-9787-20d9bf00490d None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:05.924775 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e671f43-db24-47bb-9787-20d9bf00490d None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:05.925216 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e671f43-db24-47bb-9787-20d9bf00490d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:05.928706 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309fbf1b-6301-4fb9-88c3-697984ee270c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 15:59:05.933568 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-309fbf1b-6301-4fb9-88c3-697984ee270c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 15:59:05.933568 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 15:59:05.933568 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 15:59:05.933568 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 15:59:05.933568 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 15:59:05.933568 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 15:59:05.933568 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 15:59:05.933568 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 15:59:05.933568 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 15:59:05.933568 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 15:59:05.933568 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 15:59:05.933568 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 15:59:05.933568 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 15:59:05.933568 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 15:59:05.933568 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 15:59:05.933568 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 15:59:05.934441 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 15:59:05.934441 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 15:59:05.934441 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 15:59:05.934441 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 15:59:05.934441 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 15:59:05.934441 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 15:59:05.934441 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 15:59:05.934441 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 15:59:05.934441 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 15:59:05.934441 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 15:59:05.934441 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 15:59:05.934441 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 15:59:05.934441 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 15:59:05.934441 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 15:59:05.934441 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 15:59:05.934441 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 15:59:05.936040 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 15:59:05.936040 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 15:59:05.936040 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 15:59:05.936040 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 15:59:05.936040 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 15:59:05.936040 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 15:59:05.936040 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 15:59:05.936040 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 15:59:05.936040 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 15:59:05.936040 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 15:59:05.936040 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 15:59:05.936040 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 15:59:05.936040 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 37/75] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 19 15:59:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 15:59:05.941057 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309fbf1b-6301-4fb9-88c3-697984ee270c None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:06.005155 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e671f43-db24-47bb-9787-20d9bf00490d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:06.006877 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e671f43-db24-47bb-9787-20d9bf00490d None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:06.007207 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:06.016752 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:06.050811 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309fbf1b-6301-4fb9-88c3-697984ee270c None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:06.085746 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281be06b-e3f1-47d5-8684-35c22e2f6dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:06.086395 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281be06b-e3f1-47d5-8684-35c22e2f6dca None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:06.086682 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281be06b-e3f1-47d5-8684-35c22e2f6dca None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:06.086952 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281be06b-e3f1-47d5-8684-35c22e2f6dca None admin] PATH_INFO: `/v3/roles/member` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:06.097107 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281be06b-e3f1-47d5-8684-35c22e2f6dca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 15:59:06.101844 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-281be06b-e3f1-47d5-8684-35c22e2f6dca None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 15:59:06.101844 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 15:59:06.101844 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 15:59:06.101844 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 15:59:06.101844 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 15:59:06.101844 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 15:59:06.101844 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 15:59:06.101844 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 15:59:06.101844 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 15:59:06.101844 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 15:59:06.101844 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 15:59:06.101844 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 15:59:06.101844 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 15:59:06.101844 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 15:59:06.101844 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 15:59:06.101844 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 15:59:06.102898 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 15:59:06.102898 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 15:59:06.102898 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 15:59:06.102898 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 15:59:06.102898 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 15:59:06.102898 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 15:59:06.102898 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 15:59:06.102898 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 15:59:06.102898 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 15:59:06.102898 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 15:59:06.102898 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 15:59:06.102898 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 15:59:06.102898 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 15:59:06.102898 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 15:59:06.102898 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 15:59:06.102898 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 15:59:06.103770 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 15:59:06.103770 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 15:59:06.103770 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 15:59:06.103770 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 15:59:06.103770 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 15:59:06.103770 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 15:59:06.103770 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 15:59:06.103770 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 15:59:06.103770 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 15:59:06.103770 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 15:59:06.103770 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 15:59:06.103770 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 15:59:06.103770 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 38/76] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 15:59:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 15:59:06.103770 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281be06b-e3f1-47d5-8684-35c22e2f6dca None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:06.113854 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 39/77] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 19 15:59:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 15:59:06.114750 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e671f43-db24-47bb-9787-20d9bf00490d None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:06.209498 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281be06b-e3f1-47d5-8684-35c22e2f6dca None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:06.212606 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e671f43-db24-47bb-9787-20d9bf00490d None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:06.220851 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:06.232005 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:06.244474 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd323da-2029-4d5e-a50d-675e67afd2b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:06.245168 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd323da-2029-4d5e-a50d-675e67afd2b3 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:06.245456 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd323da-2029-4d5e-a50d-675e67afd2b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:06.245735 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd323da-2029-4d5e-a50d-675e67afd2b3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:06.248660 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0823406c-f303-44d9-a31b-56e9db3baa55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:06.249265 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0823406c-f303-44d9-a31b-56e9db3baa55 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:06.249695 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0823406c-f303-44d9-a31b-56e9db3baa55 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:06.250520 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0823406c-f303-44d9-a31b-56e9db3baa55 None admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:06.251185 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0823406c-f303-44d9-a31b-56e9db3baa55 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:06.251494 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0823406c-f303-44d9-a31b-56e9db3baa55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:06.252884 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0823406c-f303-44d9-a31b-56e9db3baa55 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:06.259060 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd323da-2029-4d5e-a50d-675e67afd2b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 15:59:06.263807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-5fd323da-2029-4d5e-a50d-675e67afd2b3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 15:59:06.263807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 15:59:06.263807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 15:59:06.263807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 15:59:06.263807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 15:59:06.263807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 15:59:06.263807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 15:59:06.263807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 15:59:06.263807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 15:59:06.263807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 15:59:06.263807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 15:59:06.263807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 15:59:06.263807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 15:59:06.263807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 15:59:06.263807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 15:59:06.263807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 15:59:06.266900 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 15:59:06.266900 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 15:59:06.266900 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 15:59:06.266900 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 15:59:06.266900 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 15:59:06.266900 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 15:59:06.266900 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 15:59:06.266900 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 15:59:06.266900 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 15:59:06.266900 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 15:59:06.266900 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 15:59:06.266900 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 15:59:06.266900 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 15:59:06.266900 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 15:59:06.266900 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 15:59:06.266900 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 15:59:06.268322 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 15:59:06.268322 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 15:59:06.268322 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 15:59:06.268322 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 15:59:06.268322 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 15:59:06.268322 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 15:59:06.268322 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 15:59:06.268322 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 15:59:06.268322 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 15:59:06.268322 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 15:59:06.268322 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 15:59:06.268322 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 15:59:06.268322 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 39/78] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Sep 19 15:59:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 15:59:06.268322 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd323da-2029-4d5e-a50d-675e67afd2b3 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:06.269734 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 40/79] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 15:59:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:06.270630 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0823406c-f303-44d9-a31b-56e9db3baa55 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:06.369077 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd323da-2029-4d5e-a50d-675e67afd2b3 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:06.370490 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0823406c-f303-44d9-a31b-56e9db3baa55 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:06.403070 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be59f51-ad13-493d-8b1f-a2c770e8e4b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:06.403709 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be59f51-ad13-493d-8b1f-a2c770e8e4b9 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:06.403813 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f81071-19df-4087-bc11-0579b9c12c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:06.403993 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be59f51-ad13-493d-8b1f-a2c770e8e4b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:06.404272 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be59f51-ad13-493d-8b1f-a2c770e8e4b9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:06.404392 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f81071-19df-4087-bc11-0579b9c12c24 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:06.404625 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f81071-19df-4087-bc11-0579b9c12c24 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:06.404902 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f81071-19df-4087-bc11-0579b9c12c24 None admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:06.405432 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be59f51-ad13-493d-8b1f-a2c770e8e4b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:06.405578 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f81071-19df-4087-bc11-0579b9c12c24 None admin] RBAC: Adding query filter params (name=member) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:06.405868 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f81071-19df-4087-bc11-0579b9c12c24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:06.407553 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be59f51-ad13-493d-8b1f-a2c770e8e4b9 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:06.407821 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f81071-19df-4087-bc11-0579b9c12c24 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:06.409122 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 40/80] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 15:59:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:06.409902 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be59f51-ad13-493d-8b1f-a2c770e8e4b9 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:06.417007 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 41/81] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Sep 19 15:59:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:06.417776 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f81071-19df-4087-bc11-0579b9c12c24 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:06.448464 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:06.460232 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:06.509277 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be59f51-ad13-493d-8b1f-a2c770e8e4b9 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:06.518019 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f81071-19df-4087-bc11-0579b9c12c24 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:06.545643 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f678982c-57de-44a1-b01c-26eb5b6b94cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:06.547589 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f678982c-57de-44a1-b01c-26eb5b6b94cc None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:06.547589 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f678982c-57de-44a1-b01c-26eb5b6b94cc None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:06.547589 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f678982c-57de-44a1-b01c-26eb5b6b94cc None admin] PATH_INFO: `/v3/projects/ad7bddf539af4f8dabada63995dab07f` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:06.548341 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f678982c-57de-44a1-b01c-26eb5b6b94cc None admin] RBAC: Authorizing `identity:get_project(project_id=ad7bddf539af4f8dabada63995dab07f)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:06.549792 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f678982c-57de-44a1-b01c-26eb5b6b94cc None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:06.553009 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 41/82] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 19 15:59:06 2022] GET /identity/v3/projects/ad7bddf539af4f8dabada63995dab07f => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:06.553009 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f678982c-57de-44a1-b01c-26eb5b6b94cc None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:06.553193 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3edcc96-12a9-48f6-b907-a3b2537e6082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:06.553193 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3edcc96-12a9-48f6-b907-a3b2537e6082 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:06.553193 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3edcc96-12a9-48f6-b907-a3b2537e6082 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:06.553532 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3edcc96-12a9-48f6-b907-a3b2537e6082 None admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:06.554116 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3edcc96-12a9-48f6-b907-a3b2537e6082 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:06.554462 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3edcc96-12a9-48f6-b907-a3b2537e6082 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:06.558047 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3edcc96-12a9-48f6-b907-a3b2537e6082 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:06.564728 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 42/83] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 19 15:59:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:06.565538 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3edcc96-12a9-48f6-b907-a3b2537e6082 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:06.653008 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f678982c-57de-44a1-b01c-26eb5b6b94cc None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:06.657278 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:06.667621 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3edcc96-12a9-48f6-b907-a3b2537e6082 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:06.670503 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:06.687985 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82de4b33-99fb-4ca8-af29-8a1a9ee81b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:06.688622 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82de4b33-99fb-4ca8-af29-8a1a9ee81b29 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:06.688903 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82de4b33-99fb-4ca8-af29-8a1a9ee81b29 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:06.689182 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82de4b33-99fb-4ca8-af29-8a1a9ee81b29 None admin] PATH_INFO: `/v3/users/18d4b3afb42f46999e6cf7b500999690` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:06.690384 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82de4b33-99fb-4ca8-af29-8a1a9ee81b29 None admin] RBAC: Authorizing `identity:get_user(user_id=18d4b3afb42f46999e6cf7b500999690)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:06.691851 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82de4b33-99fb-4ca8-af29-8a1a9ee81b29 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:06.693430 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 42/84] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 19 15:59:06 2022] GET /identity/v3/users/18d4b3afb42f46999e6cf7b500999690 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:06.694200 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82de4b33-99fb-4ca8-af29-8a1a9ee81b29 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:06.707126 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4d42a0-ef2e-4099-ac4b-090e55042f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:06.707737 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4d42a0-ef2e-4099-ac4b-090e55042f9a None admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:06.708015 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4d42a0-ef2e-4099-ac4b-090e55042f9a None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:06.708288 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4d42a0-ef2e-4099-ac4b-090e55042f9a None admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:06.708936 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4d42a0-ef2e-4099-ac4b-090e55042f9a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:06.710451 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4d42a0-ef2e-4099-ac4b-090e55042f9a None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:06.739623 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 43/85] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Sep 19 15:59:06 2022] POST /identity/v3/users => generated 289 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 15:59:06.739623 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4d42a0-ef2e-4099-ac4b-090e55042f9a None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:06.822835 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82de4b33-99fb-4ca8-af29-8a1a9ee81b29 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:06.865283 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2895ce35-5cbb-4a53-b56c-0a6f339c4b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:06.865969 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2895ce35-5cbb-4a53-b56c-0a6f339c4b24 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:06.866253 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2895ce35-5cbb-4a53-b56c-0a6f339c4b24 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:06.866569 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2895ce35-5cbb-4a53-b56c-0a6f339c4b24 None admin] PATH_INFO: `/v3/users/bbf2a95469bc4f3593466124dfc85117` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:06.867807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2895ce35-5cbb-4a53-b56c-0a6f339c4b24 None admin] RBAC: Authorizing `identity:get_user(user_id=bbf2a95469bc4f3593466124dfc85117)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:06.869251 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2895ce35-5cbb-4a53-b56c-0a6f339c4b24 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:06.875934 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 43/86] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 19 15:59:06 2022] GET /identity/v3/users/bbf2a95469bc4f3593466124dfc85117 => generated 275 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:06.875934 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2895ce35-5cbb-4a53-b56c-0a6f339c4b24 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:06.876198 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4d42a0-ef2e-4099-ac4b-090e55042f9a None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:06.913568 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b6415b-9d7f-454b-ba8f-4c6348eebbf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:06.914214 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b6415b-9d7f-454b-ba8f-4c6348eebbf4 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:06.914528 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b6415b-9d7f-454b-ba8f-4c6348eebbf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:06.914805 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b6415b-9d7f-454b-ba8f-4c6348eebbf4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:06.915538 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b6415b-9d7f-454b-ba8f-4c6348eebbf4 None admin] RBAC: Adding query filter params (role.id=31b0628699ea4f98824f20e7568748cc, scope.project.id=ad7bddf539af4f8dabada63995dab07f, user.id=bbf2a95469bc4f3593466124dfc85117) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:06.915851 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b6415b-9d7f-454b-ba8f-4c6348eebbf4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:06.917576 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b6415b-9d7f-454b-ba8f-4c6348eebbf4 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:06.925323 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 44/87] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 19 15:59:06 2022] GET /identity/v3/role_assignments?user.id=bbf2a95469bc4f3593466124dfc85117&scope.project.id=ad7bddf539af4f8dabada63995dab07f&role.id=31b0628699ea4f98824f20e7568748cc => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:06.926160 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b6415b-9d7f-454b-ba8f-4c6348eebbf4 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:06.935282 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:06.954448 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:07.005275 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2895ce35-5cbb-4a53-b56c-0a6f339c4b24 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:07.059457 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f7a65e-d9d6-4896-b385-17fa336af0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:07.060257 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f7a65e-d9d6-4896-b385-17fa336af0dd None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:07.060607 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f7a65e-d9d6-4896-b385-17fa336af0dd None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:07.060948 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f7a65e-d9d6-4896-b385-17fa336af0dd None admin] PATH_INFO: `/v3/users/bbf2a95469bc4f3593466124dfc85117` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:07.062885 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f7a65e-d9d6-4896-b385-17fa336af0dd None admin] RBAC: Authorizing `identity:get_user(user_id=bbf2a95469bc4f3593466124dfc85117)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:07.064482 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f7a65e-d9d6-4896-b385-17fa336af0dd None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:07.067999 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b6415b-9d7f-454b-ba8f-4c6348eebbf4 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:07.070731 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 44/88] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 19 15:59:06 2022] GET /identity/v3/users/bbf2a95469bc4f3593466124dfc85117 => generated 275 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:07.072963 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f7a65e-d9d6-4896-b385-17fa336af0dd None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:07.119877 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337c8584-2f77-4e24-8423-16ab642a3a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:07.120511 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c8584-2f77-4e24-8423-16ab642a3a1b None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:07.120788 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c8584-2f77-4e24-8423-16ab642a3a1b None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:07.121064 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c8584-2f77-4e24-8423-16ab642a3a1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:07.213372 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f7a65e-d9d6-4896-b385-17fa336af0dd None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:07.218048 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337c8584-2f77-4e24-8423-16ab642a3a1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:07.219882 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337c8584-2f77-4e24-8423-16ab642a3a1b None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:07.239283 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:07.245104 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:07.265708 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4656589b-eb57-4874-8d4b-2e0d5a37cdf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:07.266253 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4656589b-eb57-4874-8d4b-2e0d5a37cdf0 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:07.266556 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4656589b-eb57-4874-8d4b-2e0d5a37cdf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:07.266830 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4656589b-eb57-4874-8d4b-2e0d5a37cdf0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:07.367554 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4656589b-eb57-4874-8d4b-2e0d5a37cdf0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:07.369153 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4656589b-eb57-4874-8d4b-2e0d5a37cdf0 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:07.371794 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 45/89] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 19 15:59:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 15:59:07.372699 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337c8584-2f77-4e24-8423-16ab642a3a1b None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:07.541443 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 45/90] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 19 15:59:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 15:59:07.542193 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4656589b-eb57-4874-8d4b-2e0d5a37cdf0 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:07.582805 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:07.592535 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337c8584-2f77-4e24-8423-16ab642a3a1b None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:07.630775 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:07.672065 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e03537c-9ce1-424b-8c39-8150611f9be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:07.673481 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e03537c-9ce1-424b-8c39-8150611f9be4 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:07.673964 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e03537c-9ce1-424b-8c39-8150611f9be4 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:07.674474 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e03537c-9ce1-424b-8c39-8150611f9be4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:07.740554 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4656589b-eb57-4874-8d4b-2e0d5a37cdf0 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:07.772293 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e03537c-9ce1-424b-8c39-8150611f9be4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:07.774299 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e03537c-9ce1-424b-8c39-8150611f9be4 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:07.830769 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8709b342-8bb3-4b6d-b686-e4e93ab106b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:07.831734 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8709b342-8bb3-4b6d-b686-e4e93ab106b4 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:07.832717 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8709b342-8bb3-4b6d-b686-e4e93ab106b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:07.833202 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8709b342-8bb3-4b6d-b686-e4e93ab106b4 None admin] PATH_INFO: `/v3/projects/ad7bddf539af4f8dabada63995dab07f` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:07.835005 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8709b342-8bb3-4b6d-b686-e4e93ab106b4 None admin] RBAC: Authorizing `identity:get_project(project_id=ad7bddf539af4f8dabada63995dab07f)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:07.836776 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8709b342-8bb3-4b6d-b686-e4e93ab106b4 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:07.839004 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 46/91] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 19 15:59:07 2022] GET /identity/v3/projects/ad7bddf539af4f8dabada63995dab07f => generated 298 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:07.840025 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8709b342-8bb3-4b6d-b686-e4e93ab106b4 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:07.954433 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:07.967405 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8709b342-8bb3-4b6d-b686-e4e93ab106b4 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:07.986632 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 46/92] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 19 15:59:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 614 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 15:59:07.987649 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e03537c-9ce1-424b-8c39-8150611f9be4 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:08.003712 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:08.016252 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b64d31-b6e2-4de7-b7fa-024a2be30a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:08.017209 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b64d31-b6e2-4de7-b7fa-024a2be30a3c None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:08.017852 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b64d31-b6e2-4de7-b7fa-024a2be30a3c None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:08.018326 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b64d31-b6e2-4de7-b7fa-024a2be30a3c None admin] PATH_INFO: `/v3/projects/63ed7377a21c45e6aa9864e1dfaafe56` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:08.019703 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b64d31-b6e2-4de7-b7fa-024a2be30a3c None admin] RBAC: Authorizing `identity:get_project(project_id=63ed7377a21c45e6aa9864e1dfaafe56)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:08.021349 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b64d31-b6e2-4de7-b7fa-024a2be30a3c None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:08.023334 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 47/93] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 19 15:59:07 2022] GET /identity/v3/projects/63ed7377a21c45e6aa9864e1dfaafe56 => generated 312 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:08.024615 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b64d31-b6e2-4de7-b7fa-024a2be30a3c None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:08.139879 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b64d31-b6e2-4de7-b7fa-024a2be30a3c None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:08.180931 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a63abb-4eaf-4329-a768-7d206e370a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:08.181944 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a63abb-4eaf-4329-a768-7d206e370a84 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:08.182922 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a63abb-4eaf-4329-a768-7d206e370a84 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:08.183424 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a63abb-4eaf-4329-a768-7d206e370a84 None admin] PATH_INFO: `/v3/projects/ad7bddf539af4f8dabada63995dab07f` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:08.184751 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a63abb-4eaf-4329-a768-7d206e370a84 None admin] RBAC: Authorizing `identity:get_project(project_id=ad7bddf539af4f8dabada63995dab07f)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:08.186387 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a63abb-4eaf-4329-a768-7d206e370a84 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:08.188662 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 48/94] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 19 15:59:08 2022] GET /identity/v3/projects/ad7bddf539af4f8dabada63995dab07f => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:08.189735 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a63abb-4eaf-4329-a768-7d206e370a84 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:08.220756 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e03537c-9ce1-424b-8c39-8150611f9be4 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:08.224894 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:08.313808 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a63abb-4eaf-4329-a768-7d206e370a84 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:08.347199 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1044ea24-c548-4805-ae72-7316e1a756e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:08.347830 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1044ea24-c548-4805-ae72-7316e1a756e9 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:08.348208 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1044ea24-c548-4805-ae72-7316e1a756e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:08.348492 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1044ea24-c548-4805-ae72-7316e1a756e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:08.349330 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1044ea24-c548-4805-ae72-7316e1a756e9 None admin] RBAC: Adding query filter params (role.id=63d2d197bfd6497583e94bfb065ce29b, scope.project.id=ad7bddf539af4f8dabada63995dab07f, user.id=18d4b3afb42f46999e6cf7b500999690) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:08.349649 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1044ea24-c548-4805-ae72-7316e1a756e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:08.351245 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1044ea24-c548-4805-ae72-7316e1a756e9 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:08.351697 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0ee977-cd20-440b-8d46-f25441796832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:08.352518 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ee977-cd20-440b-8d46-f25441796832 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:08.353280 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ee977-cd20-440b-8d46-f25441796832 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:08.353774 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ee977-cd20-440b-8d46-f25441796832 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:08.354727 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0ee977-cd20-440b-8d46-f25441796832 None admin] RBAC: Adding query filter params (role.id=31b0628699ea4f98824f20e7568748cc, scope.project.id=63ed7377a21c45e6aa9864e1dfaafe56, user.id=bbf2a95469bc4f3593466124dfc85117) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:08.355239 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0ee977-cd20-440b-8d46-f25441796832 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:08.356945 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0ee977-cd20-440b-8d46-f25441796832 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:08.363609 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 49/95] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 19 15:59:08 2022] GET /identity/v3/role_assignments?user.id=bbf2a95469bc4f3593466124dfc85117&scope.project.id=63ed7377a21c45e6aa9864e1dfaafe56&role.id=31b0628699ea4f98824f20e7568748cc => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:08.364754 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0ee977-cd20-440b-8d46-f25441796832 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:08.372543 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 47/96] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 19 15:59:07 2022] GET /identity/v3/role_assignments?user.id=18d4b3afb42f46999e6cf7b500999690&scope.project.id=ad7bddf539af4f8dabada63995dab07f&role.id=63d2d197bfd6497583e94bfb065ce29b => generated 259 bytes in 385 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:08.470587 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:08.481131 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0ee977-cd20-440b-8d46-f25441796832 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:08.521533 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c5e0a9-9f9b-4d8b-821b-56e4de20bddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:08.522616 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c5e0a9-9f9b-4d8b-821b-56e4de20bddd None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:08.523097 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c5e0a9-9f9b-4d8b-821b-56e4de20bddd None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:08.523568 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c5e0a9-9f9b-4d8b-821b-56e4de20bddd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:08.524489 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c5e0a9-9f9b-4d8b-821b-56e4de20bddd None admin] RBAC: Adding query filter params (role.id=a1606b79a2c840238de6860bc69b49b7, scope.project.id=ad7bddf539af4f8dabada63995dab07f, user.id=bbf2a95469bc4f3593466124dfc85117) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:08.524999 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c5e0a9-9f9b-4d8b-821b-56e4de20bddd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:08.526688 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c5e0a9-9f9b-4d8b-821b-56e4de20bddd None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:08.533175 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 50/97] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 19 15:59:08 2022] GET /identity/v3/role_assignments?user.id=bbf2a95469bc4f3593466124dfc85117&scope.project.id=ad7bddf539af4f8dabada63995dab07f&role.id=a1606b79a2c840238de6860bc69b49b7 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:09.584508 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f65200-231a-4024-a98c-4964cdf49ee1 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:09.585021 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f65200-231a-4024-a98c-4964cdf49ee1 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:09.585125 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f65200-231a-4024-a98c-4964cdf49ee1 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:09.586560 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 48/98] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:59:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 15:59:09.592067 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85d4d17-24a7-4f83-8629-8e29fe5411f8 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:09.592520 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85d4d17-24a7-4f83-8629-8e29fe5411f8 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:09.592701 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85d4d17-24a7-4f83-8629-8e29fe5411f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:09.615038 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-c85d4d17-24a7-4f83-8629-8e29fe5411f8 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:59:09.616247 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-c85d4d17-24a7-4f83-8629-8e29fe5411f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:59:09.714921 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 51/99] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:59:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 15:59:09.866434 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e7bf2-4e8b-453d-ac30-5880732b7b0d None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:09.866799 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e7bf2-4e8b-453d-ac30-5880732b7b0d None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:09.867661 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e7bf2-4e8b-453d-ac30-5880732b7b0d None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:09.869728 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 49/100] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:59:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 15:59:09.876282 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e8d043-443c-45c6-ab4b-b37e27352e3b None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:09.876919 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e8d043-443c-45c6-ab4b-b37e27352e3b None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:09.877402 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e8d043-443c-45c6-ab4b-b37e27352e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:09.884287 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c37154-119d-478d-947c-2551da050dbd None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:09.884996 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c37154-119d-478d-947c-2551da050dbd None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:09.885480 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c37154-119d-478d-947c-2551da050dbd None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:09.887093 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 50/101] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:59:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 15:59:09.899986 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fdf40f-cd8a-4cba-8ec5-e95b47924357 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:09.900620 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fdf40f-cd8a-4cba-8ec5-e95b47924357 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:09.901104 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fdf40f-cd8a-4cba-8ec5-e95b47924357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:09.911488 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-d6e8d043-443c-45c6-ab4b-b37e27352e3b None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:59:09.912710 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-d6e8d043-443c-45c6-ab4b-b37e27352e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:59:09.935971 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-86fdf40f-cd8a-4cba-8ec5-e95b47924357 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:59:09.938202 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-86fdf40f-cd8a-4cba-8ec5-e95b47924357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:59:10.029298 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 52/102] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:59:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 15:59:10.099875 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 51/103] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:59:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 19 15:59:10.279610 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e8d043-443c-45c6-ab4b-b37e27352e3b None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:10.297878 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:10.402840 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e8d043-443c-45c6-ab4b-b37e27352e3b None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:10.443229 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6324576-13fb-4c76-aad5-8b31393cb455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:10.443867 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6324576-13fb-4c76-aad5-8b31393cb455 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:10.444149 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6324576-13fb-4c76-aad5-8b31393cb455 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:10.444426 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6324576-13fb-4c76-aad5-8b31393cb455 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:10.445600 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6324576-13fb-4c76-aad5-8b31393cb455 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:10.447079 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6324576-13fb-4c76-aad5-8b31393cb455 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:10.448983 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 53/104] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 15:59:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:10.454923 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fdf40f-cd8a-4cba-8ec5-e95b47924357 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:10.469342 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:10.571875 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6324576-13fb-4c76-aad5-8b31393cb455 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:10.598483 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fdf40f-cd8a-4cba-8ec5-e95b47924357 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:10.659448 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcd6b89-f7fd-4d55-ab1e-69a05701d2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:10.660074 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcd6b89-f7fd-4d55-ab1e-69a05701d2fa None admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:10.660360 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcd6b89-f7fd-4d55-ab1e-69a05701d2fa None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:10.660688 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcd6b89-f7fd-4d55-ab1e-69a05701d2fa None admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:10.661372 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcd6b89-f7fd-4d55-ab1e-69a05701d2fa None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:10.662867 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcd6b89-f7fd-4d55-ab1e-69a05701d2fa None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:10.714539 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 52/105] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Sep 19 15:59:10 2022] POST /identity/v3/projects => generated 302 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 15:59:10.715314 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcd6b89-f7fd-4d55-ab1e-69a05701d2fa None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:10.721735 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:10.791684 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6324576-13fb-4c76-aad5-8b31393cb455 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:10.883316 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:10.887340 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c870a0-c13d-4bc7-963d-755149e8479b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:10.889032 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c870a0-c13d-4bc7-963d-755149e8479b None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:10.889329 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c870a0-c13d-4bc7-963d-755149e8479b None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:10.889612 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c870a0-c13d-4bc7-963d-755149e8479b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:10.929916 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c870a0-c13d-4bc7-963d-755149e8479b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 15:59:10.936382 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-19c870a0-c13d-4bc7-963d-755149e8479b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 15:59:10.936382 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 15:59:10.936382 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 15:59:10.936382 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 15:59:10.936382 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 15:59:10.936382 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 15:59:10.936382 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 15:59:10.936382 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 15:59:10.936382 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 15:59:10.936382 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 15:59:10.936382 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 15:59:10.936382 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 15:59:10.936382 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 15:59:10.936382 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 15:59:10.936382 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 15:59:10.936382 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 15:59:10.937807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 15:59:10.937807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 15:59:10.937807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 15:59:10.937807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 15:59:10.937807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 15:59:10.937807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 15:59:10.937807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 15:59:10.937807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 15:59:10.937807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 15:59:10.937807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 15:59:10.937807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 15:59:10.937807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 15:59:10.937807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 15:59:10.937807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 15:59:10.937807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 15:59:10.937807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 15:59:10.938961 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 15:59:10.938961 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 15:59:10.938961 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 15:59:10.938961 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 15:59:10.938961 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 15:59:10.938961 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 15:59:10.938961 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 15:59:10.938961 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 15:59:10.938961 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 15:59:10.938961 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 15:59:10.938961 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 15:59:10.938961 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 15:59:10.938961 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 54/106] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 15:59:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 15:59:10.946047 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c870a0-c13d-4bc7-963d-755149e8479b None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:10.959625 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcd6b89-f7fd-4d55-ab1e-69a05701d2fa None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:11.043205 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cd3477-c5ff-4309-bc52-3caeee0cd0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:11.043884 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd3477-c5ff-4309-bc52-3caeee0cd0ec None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:11.044175 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd3477-c5ff-4309-bc52-3caeee0cd0ec None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:11.044464 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd3477-c5ff-4309-bc52-3caeee0cd0ec None admin] PATH_INFO: `/v3/roles/member` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:11.060369 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c870a0-c13d-4bc7-963d-755149e8479b None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:11.072126 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cd3477-c5ff-4309-bc52-3caeee0cd0ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 15:59:11.077626 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-16cd3477-c5ff-4309-bc52-3caeee0cd0ec None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 15:59:11.077626 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 15:59:11.077626 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 15:59:11.077626 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 15:59:11.077626 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 15:59:11.077626 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 15:59:11.077626 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 15:59:11.077626 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 15:59:11.077626 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 15:59:11.077626 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 15:59:11.077626 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 15:59:11.077626 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 15:59:11.077626 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 15:59:11.077626 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 15:59:11.077626 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 15:59:11.077626 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 15:59:11.078513 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 15:59:11.078513 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 15:59:11.078513 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 15:59:11.078513 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 15:59:11.078513 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 15:59:11.078513 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 15:59:11.078513 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 15:59:11.078513 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 15:59:11.078513 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 15:59:11.078513 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 15:59:11.078513 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 15:59:11.078513 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 15:59:11.078513 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 15:59:11.078513 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 15:59:11.078513 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 15:59:11.078513 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 15:59:11.079257 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 15:59:11.079257 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 15:59:11.079257 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 15:59:11.079257 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 15:59:11.079257 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 15:59:11.079257 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 15:59:11.079257 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 15:59:11.079257 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 15:59:11.079257 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 15:59:11.079257 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 15:59:11.079257 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 15:59:11.079257 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 15:59:11.079257 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 53/107] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 15:59:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 15:59:11.084683 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cd3477-c5ff-4309-bc52-3caeee0cd0ec None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:11.109184 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2172e98e-07f8-4035-bde7-316e31f683e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:11.110193 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2172e98e-07f8-4035-bde7-316e31f683e2 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:11.110706 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2172e98e-07f8-4035-bde7-316e31f683e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:11.111175 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2172e98e-07f8-4035-bde7-316e31f683e2 None admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:11.112070 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2172e98e-07f8-4035-bde7-316e31f683e2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:11.112569 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2172e98e-07f8-4035-bde7-316e31f683e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:11.114183 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2172e98e-07f8-4035-bde7-316e31f683e2 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:11.132107 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 55/108] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Sep 19 15:59:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 15:59:11.141152 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2172e98e-07f8-4035-bde7-316e31f683e2 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:11.146937 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:11.336698 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cd3477-c5ff-4309-bc52-3caeee0cd0ec None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:11.340831 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2172e98e-07f8-4035-bde7-316e31f683e2 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:11.373089 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:11.404116 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a8f8fa-bd11-4c7d-a005-afa6973f84f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:11.405082 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a8f8fa-bd11-4c7d-a005-afa6973f84f1 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:11.405558 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a8f8fa-bd11-4c7d-a005-afa6973f84f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:11.406072 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a8f8fa-bd11-4c7d-a005-afa6973f84f1 None admin] PATH_INFO: `/v3/users/3a951e2fa39c4e7c8305f6e6067c55f4` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:11.412654 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075a8f0f-2f21-4cf7-8369-01f968016cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:11.413274 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075a8f0f-2f21-4cf7-8369-01f968016cca None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:11.413559 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075a8f0f-2f21-4cf7-8369-01f968016cca None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:11.413839 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075a8f0f-2f21-4cf7-8369-01f968016cca None admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:11.415104 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075a8f0f-2f21-4cf7-8369-01f968016cca None admin] RBAC: Adding query filter params (name=member) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:11.415104 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075a8f0f-2f21-4cf7-8369-01f968016cca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:11.416508 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075a8f0f-2f21-4cf7-8369-01f968016cca None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:11.430552 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 54/109] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Sep 19 15:59:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:11.432938 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075a8f0f-2f21-4cf7-8369-01f968016cca None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:11.434598 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a8f8fa-bd11-4c7d-a005-afa6973f84f1 None admin] RBAC: Authorizing `identity:get_user(user_id=3a951e2fa39c4e7c8305f6e6067c55f4)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:11.436316 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a8f8fa-bd11-4c7d-a005-afa6973f84f1 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:11.438931 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 56/110] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 19 15:59:11 2022] GET /identity/v3/users/3a951e2fa39c4e7c8305f6e6067c55f4 => generated 289 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:11.444835 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a8f8fa-bd11-4c7d-a005-afa6973f84f1 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:11.549500 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:11.606977 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a8f8fa-bd11-4c7d-a005-afa6973f84f1 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:11.637886 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075a8f0f-2f21-4cf7-8369-01f968016cca None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:11.671206 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6450b13a-c895-4bf7-a287-d0310cb87cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:11.672210 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6450b13a-c895-4bf7-a287-d0310cb87cd8 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:11.672695 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6450b13a-c895-4bf7-a287-d0310cb87cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:11.673163 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6450b13a-c895-4bf7-a287-d0310cb87cd8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:11.687164 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b768c70-551e-469f-b504-68815b7dca4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:11.687830 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b768c70-551e-469f-b504-68815b7dca4e None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:11.688120 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b768c70-551e-469f-b504-68815b7dca4e None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:11.688403 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b768c70-551e-469f-b504-68815b7dca4e None admin] PATH_INFO: `/v3/users/bbf2a95469bc4f3593466124dfc85117` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:11.689962 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b768c70-551e-469f-b504-68815b7dca4e None admin] RBAC: Authorizing `identity:get_user(user_id=bbf2a95469bc4f3593466124dfc85117)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:11.691175 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b768c70-551e-469f-b504-68815b7dca4e None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:11.693119 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 55/111] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 19 15:59:11 2022] GET /identity/v3/users/bbf2a95469bc4f3593466124dfc85117 => generated 275 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:11.698652 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b768c70-551e-469f-b504-68815b7dca4e None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:11.798286 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:11.807273 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6450b13a-c895-4bf7-a287-d0310cb87cd8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:11.809427 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6450b13a-c895-4bf7-a287-d0310cb87cd8 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:11.854179 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b768c70-551e-469f-b504-68815b7dca4e None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:11.894268 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:11.895793 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c07c4f0-776e-4c9c-ba34-c7c82c8c3bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:11.896668 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c07c4f0-776e-4c9c-ba34-c7c82c8c3bed None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:11.897130 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c07c4f0-776e-4c9c-ba34-c7c82c8c3bed None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:11.897551 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c07c4f0-776e-4c9c-ba34-c7c82c8c3bed None admin] PATH_INFO: `/v3/projects/ad7bddf539af4f8dabada63995dab07f` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:11.898927 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c07c4f0-776e-4c9c-ba34-c7c82c8c3bed None admin] RBAC: Authorizing `identity:get_project(project_id=ad7bddf539af4f8dabada63995dab07f)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:11.900529 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c07c4f0-776e-4c9c-ba34-c7c82c8c3bed None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:11.902679 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 56/112] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 19 15:59:11 2022] GET /identity/v3/projects/ad7bddf539af4f8dabada63995dab07f => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:11.909144 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9957e690-3df0-4ba3-b42a-783f78f3b7ac None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:11.909144 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9957e690-3df0-4ba3-b42a-783f78f3b7ac None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:11.909144 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9957e690-3df0-4ba3-b42a-783f78f3b7ac None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:11.909144 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 57/113] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:59:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 15:59:11.909443 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6503dae-0aec-4e52-87ae-328d1fffd261 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:11.910150 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6503dae-0aec-4e52-87ae-328d1fffd261 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:11.910575 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6503dae-0aec-4e52-87ae-328d1fffd261 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:11.912309 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 58/114] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:59:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 15:59:11.913229 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6503dae-0aec-4e52-87ae-328d1fffd261 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:11.956695 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 57/115] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 19 15:59:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 19 15:59:11.962660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21ef19-4e7e-4547-8d3c-a663ca2a053e None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:11.962660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21ef19-4e7e-4547-8d3c-a663ca2a053e None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:11.962660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21ef19-4e7e-4547-8d3c-a663ca2a053e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:11.985779 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-de21ef19-4e7e-4547-8d3c-a663ca2a053e None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:59:11.987061 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-de21ef19-4e7e-4547-8d3c-a663ca2a053e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:59:12.041925 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6503dae-0aec-4e52-87ae-328d1fffd261 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:12.061428 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:12.068217 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 58/116] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:59:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 15:59:12.068217 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fadcd8-7517-4957-8fe9-daf884da9b31 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:12.068217 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fadcd8-7517-4957-8fe9-daf884da9b31 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:12.068217 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fadcd8-7517-4957-8fe9-daf884da9b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:12.086527 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2b055e-265e-48b0-ab8e-521093b7c8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:12.087202 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2b055e-265e-48b0-ab8e-521093b7c8fe None admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:12.087486 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2b055e-265e-48b0-ab8e-521093b7c8fe None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:12.087765 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2b055e-265e-48b0-ab8e-521093b7c8fe None admin] PATH_INFO: `/v3/projects/ad7bddf539af4f8dabada63995dab07f/users/bbf2a95469bc4f3593466124dfc85117/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:12.088377 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-94fadcd8-7517-4957-8fe9-daf884da9b31 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:59:12.089799 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-94fadcd8-7517-4957-8fe9-daf884da9b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:59:12.089896 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2b055e-265e-48b0-ab8e-521093b7c8fe None admin] RBAC: Authorizing `identity:create_grant(project_id=ad7bddf539af4f8dabada63995dab07f, user_id=bbf2a95469bc4f3593466124dfc85117, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:12.091506 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2b055e-265e-48b0-ab8e-521093b7c8fe None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:12.110888 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 59/117] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Sep 19 15:59:11 2022] PUT /identity/v3/projects/ad7bddf539af4f8dabada63995dab07f/users/bbf2a95469bc4f3593466124dfc85117/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:59:12.112605 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2b055e-265e-48b0-ab8e-521093b7c8fe None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:12.256178 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 59/118] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:59:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 15:59:12.257412 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c13a10-1b7b-4cce-9b7b-8b74eec12c9a None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:12.257704 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c13a10-1b7b-4cce-9b7b-8b74eec12c9a None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:12.258046 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c13a10-1b7b-4cce-9b7b-8b74eec12c9a None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:12.265081 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 60/119] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:59:12 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 15:59:12.271241 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d6021-cf17-4199-8f0e-1e5b5082551a None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:12.271572 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d6021-cf17-4199-8f0e-1e5b5082551a None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:12.271809 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d6021-cf17-4199-8f0e-1e5b5082551a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:12.303425 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-4f5d6021-cf17-4199-8f0e-1e5b5082551a None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:59:12.304735 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-4f5d6021-cf17-4199-8f0e-1e5b5082551a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:59:12.337301 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:12.366222 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2b055e-265e-48b0-ab8e-521093b7c8fe None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:12.417040 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21fc073-8f1c-40b9-8bfd-8f482bfbe761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:12.417694 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21fc073-8f1c-40b9-8bfd-8f482bfbe761 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:12.417979 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21fc073-8f1c-40b9-8bfd-8f482bfbe761 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:12.418258 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21fc073-8f1c-40b9-8bfd-8f482bfbe761 None admin] PATH_INFO: `/v3/projects/ad7bddf539af4f8dabada63995dab07f` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:12.420156 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21fc073-8f1c-40b9-8bfd-8f482bfbe761 None admin] RBAC: Authorizing `identity:get_project(project_id=ad7bddf539af4f8dabada63995dab07f)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:12.421674 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21fc073-8f1c-40b9-8bfd-8f482bfbe761 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:12.423381 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 60/120] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 19 15:59:12 2022] GET /identity/v3/projects/ad7bddf539af4f8dabada63995dab07f => generated 298 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:12.424182 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21fc073-8f1c-40b9-8bfd-8f482bfbe761 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:12.427774 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 61/121] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:59:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 15:59:12.429309 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5d6021-cf17-4199-8f0e-1e5b5082551a None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:12.498890 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:12.550853 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5d6021-cf17-4199-8f0e-1e5b5082551a None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:12.598444 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec859c76-508b-4978-9f03-954c9252bb1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:12.599495 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec859c76-508b-4978-9f03-954c9252bb1d None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:12.599990 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec859c76-508b-4978-9f03-954c9252bb1d None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:12.600455 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec859c76-508b-4978-9f03-954c9252bb1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:12.601549 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec859c76-508b-4978-9f03-954c9252bb1d None admin] RBAC: Adding query filter params (role.id=625d911d4d934c1e80f0a57620077d49, scope.project.id=ad7bddf539af4f8dabada63995dab07f, user.id=3a951e2fa39c4e7c8305f6e6067c55f4) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:12.602189 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec859c76-508b-4978-9f03-954c9252bb1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:12.603905 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec859c76-508b-4978-9f03-954c9252bb1d None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:12.605162 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21fc073-8f1c-40b9-8bfd-8f482bfbe761 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:12.611012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 62/122] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 19 15:59:12 2022] GET /identity/v3/role_assignments?user.id=3a951e2fa39c4e7c8305f6e6067c55f4&scope.project.id=ad7bddf539af4f8dabada63995dab07f&role.id=625d911d4d934c1e80f0a57620077d49 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:12.665225 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a983a76e-e177-4bc8-97db-609c3184928e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:12.666928 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a983a76e-e177-4bc8-97db-609c3184928e None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:12.667401 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a983a76e-e177-4bc8-97db-609c3184928e None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:12.668003 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a983a76e-e177-4bc8-97db-609c3184928e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:12.685309 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a983a76e-e177-4bc8-97db-609c3184928e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 15:59:12.692844 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-a983a76e-e177-4bc8-97db-609c3184928e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 15:59:12.692844 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 15:59:12.692844 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 15:59:12.692844 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 15:59:12.692844 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 15:59:12.692844 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 15:59:12.692844 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 15:59:12.692844 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 15:59:12.692844 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 15:59:12.692844 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 15:59:12.692844 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 15:59:12.692844 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 15:59:12.692844 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 15:59:12.692844 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 15:59:12.692844 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 15:59:12.692844 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 15:59:12.693763 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 15:59:12.693763 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 15:59:12.693763 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 15:59:12.693763 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 15:59:12.693763 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 15:59:12.693763 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 15:59:12.693763 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 15:59:12.693763 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 15:59:12.693763 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 15:59:12.693763 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 15:59:12.693763 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 15:59:12.693763 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 15:59:12.693763 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 15:59:12.693763 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 15:59:12.693763 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 15:59:12.693763 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 15:59:12.694529 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 15:59:12.694529 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 15:59:12.694529 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 15:59:12.694529 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 15:59:12.694529 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 15:59:12.694529 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 15:59:12.694529 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 15:59:12.694529 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 15:59:12.694529 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 15:59:12.694529 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 15:59:12.694529 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 15:59:12.694529 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 15:59:12.701515 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 61/123] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 19 15:59:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 15:59:12.703483 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec859c76-508b-4978-9f03-954c9252bb1d None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:12.724106 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a983a76e-e177-4bc8-97db-609c3184928e None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:12.736041 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:12.926692 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec859c76-508b-4978-9f03-954c9252bb1d None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:12.982702 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a983a76e-e177-4bc8-97db-609c3184928e None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:12.989549 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:13.015215 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fc5e38-4cdf-4e1c-a7ba-cf6f79fcadc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:13.015843 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc5e38-4cdf-4e1c-a7ba-cf6f79fcadc8 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:13.016126 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc5e38-4cdf-4e1c-a7ba-cf6f79fcadc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:13.016408 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc5e38-4cdf-4e1c-a7ba-cf6f79fcadc8 None admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:13.017088 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fc5e38-4cdf-4e1c-a7ba-cf6f79fcadc8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:13.017397 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fc5e38-4cdf-4e1c-a7ba-cf6f79fcadc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:13.019763 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fc5e38-4cdf-4e1c-a7ba-cf6f79fcadc8 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:13.039781 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 63/124] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 15:59:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:13.048527 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fc5e38-4cdf-4e1c-a7ba-cf6f79fcadc8 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:13.074704 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ac2a36-32a4-4aac-80c7-8c4cc039a61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:13.075414 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ac2a36-32a4-4aac-80c7-8c4cc039a61e None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:13.075696 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ac2a36-32a4-4aac-80c7-8c4cc039a61e None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:13.075973 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ac2a36-32a4-4aac-80c7-8c4cc039a61e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:13.107438 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ac2a36-32a4-4aac-80c7-8c4cc039a61e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 15:59:13.120213 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-71ac2a36-32a4-4aac-80c7-8c4cc039a61e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 15:59:13.120213 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 15:59:13.120213 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 15:59:13.120213 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 15:59:13.120213 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 15:59:13.120213 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 15:59:13.120213 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 15:59:13.120213 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 15:59:13.120213 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 15:59:13.120213 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 15:59:13.120213 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 15:59:13.120213 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 15:59:13.120213 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 15:59:13.120213 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 15:59:13.120213 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 15:59:13.120213 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 15:59:13.121089 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 15:59:13.121089 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 15:59:13.121089 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 15:59:13.121089 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 15:59:13.121089 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 15:59:13.121089 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 15:59:13.121089 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 15:59:13.121089 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 15:59:13.121089 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 15:59:13.121089 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 15:59:13.121089 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 15:59:13.121089 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 15:59:13.121089 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 15:59:13.121089 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 15:59:13.121089 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 15:59:13.121089 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 15:59:13.121834 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 15:59:13.121834 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 15:59:13.121834 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 15:59:13.121834 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 15:59:13.121834 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 15:59:13.121834 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 15:59:13.121834 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 15:59:13.121834 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 15:59:13.121834 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 15:59:13.121834 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 15:59:13.121834 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 15:59:13.121834 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 15:59:13.123891 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 62/125] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Sep 19 15:59:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 399 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 15:59:13.153936 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ac2a36-32a4-4aac-80c7-8c4cc039a61e None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:13.228449 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:13.282511 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fc5e38-4cdf-4e1c-a7ba-cf6f79fcadc8 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:13.309912 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ac2a36-32a4-4aac-80c7-8c4cc039a61e None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:13.354398 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879a45a1-ef2b-4320-b5e5-9e860508cbce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:13.354398 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879a45a1-ef2b-4320-b5e5-9e860508cbce None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:13.354398 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879a45a1-ef2b-4320-b5e5-9e860508cbce None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:13.354398 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879a45a1-ef2b-4320-b5e5-9e860508cbce None admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:13.354786 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879a45a1-ef2b-4320-b5e5-9e860508cbce None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:13.354786 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879a45a1-ef2b-4320-b5e5-9e860508cbce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:13.354786 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879a45a1-ef2b-4320-b5e5-9e860508cbce None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:13.361624 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba92708-29a1-4a97-9c7c-88efb27b8369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:13.362581 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba92708-29a1-4a97-9c7c-88efb27b8369 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:13.363057 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba92708-29a1-4a97-9c7c-88efb27b8369 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:13.363524 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba92708-29a1-4a97-9c7c-88efb27b8369 None admin] PATH_INFO: `/v3/users/18d4b3afb42f46999e6cf7b500999690` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:13.364272 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 63/126] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 19 15:59:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:13.365070 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba92708-29a1-4a97-9c7c-88efb27b8369 None admin] RBAC: Authorizing `identity:get_user(user_id=18d4b3afb42f46999e6cf7b500999690)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:13.373517 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba92708-29a1-4a97-9c7c-88efb27b8369 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:13.373702 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879a45a1-ef2b-4320-b5e5-9e860508cbce None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:13.373795 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 64/127] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 19 15:59:13 2022] GET /identity/v3/users/18d4b3afb42f46999e6cf7b500999690 => generated 247 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:13.373795 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba92708-29a1-4a97-9c7c-88efb27b8369 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:13.415882 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:13.498742 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879a45a1-ef2b-4320-b5e5-9e860508cbce None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:13.537665 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75319fd-60b3-47c7-a226-8a8a13772146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:13.538310 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75319fd-60b3-47c7-a226-8a8a13772146 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:13.538627 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75319fd-60b3-47c7-a226-8a8a13772146 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:13.538908 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75319fd-60b3-47c7-a226-8a8a13772146 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:13.552016 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75319fd-60b3-47c7-a226-8a8a13772146 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 15:59:13.557672 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-c75319fd-60b3-47c7-a226-8a8a13772146 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 15:59:13.557672 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 15:59:13.557672 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 15:59:13.557672 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 15:59:13.557672 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 15:59:13.557672 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 15:59:13.557672 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 15:59:13.557672 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 15:59:13.557672 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 15:59:13.557672 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 15:59:13.557672 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 15:59:13.557672 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 15:59:13.557672 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 15:59:13.557672 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 15:59:13.557672 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 15:59:13.557672 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 15:59:13.558546 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 15:59:13.558546 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 15:59:13.558546 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 15:59:13.558546 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 15:59:13.558546 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 15:59:13.558546 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 15:59:13.558546 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 15:59:13.558546 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 15:59:13.558546 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 15:59:13.558546 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 15:59:13.558546 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 15:59:13.558546 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 15:59:13.558546 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 15:59:13.558546 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 15:59:13.558546 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 15:59:13.558546 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 15:59:13.559321 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 15:59:13.559321 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 15:59:13.559321 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 15:59:13.559321 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 15:59:13.559321 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 15:59:13.559321 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 15:59:13.559321 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 15:59:13.559321 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 15:59:13.559321 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 15:59:13.559321 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 15:59:13.559321 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 15:59:13.559321 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 15:59:13.559321 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 64/128] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 15:59:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 15:59:13.563528 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75319fd-60b3-47c7-a226-8a8a13772146 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:13.686913 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75319fd-60b3-47c7-a226-8a8a13772146 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:13.698720 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:13.708341 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba92708-29a1-4a97-9c7c-88efb27b8369 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:13.726753 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22092fb1-f235-445c-9883-50be1c2c4342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:13.727401 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22092fb1-f235-445c-9883-50be1c2c4342 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:13.727681 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22092fb1-f235-445c-9883-50be1c2c4342 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:13.727955 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22092fb1-f235-445c-9883-50be1c2c4342 None admin] PATH_INFO: `/v3/projects/ad7bddf539af4f8dabada63995dab07f` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:13.729644 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22092fb1-f235-445c-9883-50be1c2c4342 None admin] RBAC: Authorizing `identity:get_project(project_id=ad7bddf539af4f8dabada63995dab07f)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:13.731223 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22092fb1-f235-445c-9883-50be1c2c4342 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:13.733534 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 65/129] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 19 15:59:13 2022] GET /identity/v3/projects/ad7bddf539af4f8dabada63995dab07f => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:13.735212 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22092fb1-f235-445c-9883-50be1c2c4342 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:13.776422 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af75b0c1-6753-4501-93cf-1d26be938d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:13.776994 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af75b0c1-6753-4501-93cf-1d26be938d96 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:13.777275 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af75b0c1-6753-4501-93cf-1d26be938d96 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:13.777557 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af75b0c1-6753-4501-93cf-1d26be938d96 None admin] PATH_INFO: `/v3/users/bbf2a95469bc4f3593466124dfc85117` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:13.778760 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af75b0c1-6753-4501-93cf-1d26be938d96 None admin] RBAC: Authorizing `identity:get_user(user_id=bbf2a95469bc4f3593466124dfc85117)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:13.780581 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af75b0c1-6753-4501-93cf-1d26be938d96 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:13.783629 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 65/130] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 19 15:59:13 2022] GET /identity/v3/users/bbf2a95469bc4f3593466124dfc85117 => generated 275 bytes in 411 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:13.784543 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af75b0c1-6753-4501-93cf-1d26be938d96 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:13.808310 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:13.864097 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22092fb1-f235-445c-9883-50be1c2c4342 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:13.913387 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9dde19-dfe0-43d4-8b24-ef9571c265d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:13.914657 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9dde19-dfe0-43d4-8b24-ef9571c265d7 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:13.915106 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9dde19-dfe0-43d4-8b24-ef9571c265d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:13.915538 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9dde19-dfe0-43d4-8b24-ef9571c265d7 None admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:13.916999 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9dde19-dfe0-43d4-8b24-ef9571c265d7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:13.917493 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9dde19-dfe0-43d4-8b24-ef9571c265d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:13.919953 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9dde19-dfe0-43d4-8b24-ef9571c265d7 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:13.928869 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af75b0c1-6753-4501-93cf-1d26be938d96 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:13.933307 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 66/131] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Sep 19 15:59:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:13.934789 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f2b928-a5ee-4015-ba3d-705c6f831b55 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:13.935239 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f2b928-a5ee-4015-ba3d-705c6f831b55 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:13.935660 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f2b928-a5ee-4015-ba3d-705c6f831b55 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:13.937122 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 67/132] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:59:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 15:59:13.937976 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f2b928-a5ee-4015-ba3d-705c6f831b55 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:13.964177 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:13.973355 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9dcee3-b70d-4cfc-b25a-eb525e3c7a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:13.974115 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9dcee3-b70d-4cfc-b25a-eb525e3c7a24 None admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:13.974498 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9dcee3-b70d-4cfc-b25a-eb525e3c7a24 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:13.974872 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9dcee3-b70d-4cfc-b25a-eb525e3c7a24 None admin] PATH_INFO: `/v3/projects/ad7bddf539af4f8dabada63995dab07f/users/18d4b3afb42f46999e6cf7b500999690/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:13.977697 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9dcee3-b70d-4cfc-b25a-eb525e3c7a24 None admin] RBAC: Authorizing `identity:create_grant(project_id=ad7bddf539af4f8dabada63995dab07f, user_id=18d4b3afb42f46999e6cf7b500999690, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:13.979319 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9dcee3-b70d-4cfc-b25a-eb525e3c7a24 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:14.000598 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 66/133] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Sep 19 15:59:13 2022] PUT /identity/v3/projects/ad7bddf539af4f8dabada63995dab07f/users/18d4b3afb42f46999e6cf7b500999690/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:59:14.006518 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a82a93-4232-4f71-a2bf-f572f376b3db None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:14.006518 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a82a93-4232-4f71-a2bf-f572f376b3db None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:14.006518 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a82a93-4232-4f71-a2bf-f572f376b3db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:14.024215 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-03a82a93-4232-4f71-a2bf-f572f376b3db None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:59:14.025557 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-03a82a93-4232-4f71-a2bf-f572f376b3db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:59:14.108710 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 67/134] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:59:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 15:59:14.110881 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a82a93-4232-4f71-a2bf-f572f376b3db None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:14.114734 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:14.130276 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f2b928-a5ee-4015-ba3d-705c6f831b55 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:14.243744 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd7b3ef-09b4-4afa-a556-9f26d1d79a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:14.244426 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd7b3ef-09b4-4afa-a556-9f26d1d79a98 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:14.244714 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd7b3ef-09b4-4afa-a556-9f26d1d79a98 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:14.244999 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd7b3ef-09b4-4afa-a556-9f26d1d79a98 None admin] PATH_INFO: `/v3/projects/ad7bddf539af4f8dabada63995dab07f` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:14.246160 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd7b3ef-09b4-4afa-a556-9f26d1d79a98 None admin] RBAC: Authorizing `identity:get_project(project_id=ad7bddf539af4f8dabada63995dab07f)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:14.247674 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd7b3ef-09b4-4afa-a556-9f26d1d79a98 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:14.254461 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 68/135] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 19 15:59:13 2022] GET /identity/v3/projects/ad7bddf539af4f8dabada63995dab07f => generated 298 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:14.259402 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd7b3ef-09b4-4afa-a556-9f26d1d79a98 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:14.324225 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a82a93-4232-4f71-a2bf-f572f376b3db None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:14.391933 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4126cc2-cddf-414c-92b8-e217899133e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:14.392909 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4126cc2-cddf-414c-92b8-e217899133e8 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:14.393413 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4126cc2-cddf-414c-92b8-e217899133e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:14.393887 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4126cc2-cddf-414c-92b8-e217899133e8 None admin] PATH_INFO: `/v3/users/bbf2a95469bc4f3593466124dfc85117` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:14.395555 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4126cc2-cddf-414c-92b8-e217899133e8 None admin] RBAC: Authorizing `identity:get_user(user_id=bbf2a95469bc4f3593466124dfc85117)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:14.398533 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4126cc2-cddf-414c-92b8-e217899133e8 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:14.401348 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 68/136] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 19 15:59:14 2022] GET /identity/v3/users/bbf2a95469bc4f3593466124dfc85117 => generated 275 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:14.407118 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4126cc2-cddf-414c-92b8-e217899133e8 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:14.427971 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:14.459172 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd7b3ef-09b4-4afa-a556-9f26d1d79a98 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:14.523044 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:14.540083 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4bb791-7c1b-46c9-80c1-4cbe8fae9661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:14.541450 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4bb791-7c1b-46c9-80c1-4cbe8fae9661 None admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:14.541986 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4bb791-7c1b-46c9-80c1-4cbe8fae9661 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:14.542505 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4bb791-7c1b-46c9-80c1-4cbe8fae9661 None admin] PATH_INFO: `/v3/projects/ad7bddf539af4f8dabada63995dab07f/users/bbf2a95469bc4f3593466124dfc85117/roles/a1606b79a2c840238de6860bc69b49b7` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:14.551185 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4bb791-7c1b-46c9-80c1-4cbe8fae9661 None admin] RBAC: Authorizing `identity:create_grant(project_id=ad7bddf539af4f8dabada63995dab07f, user_id=bbf2a95469bc4f3593466124dfc85117, role_id=a1606b79a2c840238de6860bc69b49b7)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:14.552990 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4bb791-7c1b-46c9-80c1-4cbe8fae9661 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:14.558411 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4126cc2-cddf-414c-92b8-e217899133e8 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:14.580114 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 69/137] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Sep 19 15:59:14 2022] PUT /identity/v3/projects/ad7bddf539af4f8dabada63995dab07f/users/bbf2a95469bc4f3593466124dfc85117/roles/a1606b79a2c840238de6860bc69b49b7 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:59:14.581415 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4bb791-7c1b-46c9-80c1-4cbe8fae9661 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:14.598720 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c301e901-d232-4def-be7e-a36cf4a6d84b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:14.598720 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c301e901-d232-4def-be7e-a36cf4a6d84b None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:14.598720 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c301e901-d232-4def-be7e-a36cf4a6d84b None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:14.598720 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c301e901-d232-4def-be7e-a36cf4a6d84b None admin] PATH_INFO: `/v3/projects/63ed7377a21c45e6aa9864e1dfaafe56` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:14.599327 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c301e901-d232-4def-be7e-a36cf4a6d84b None admin] RBAC: Authorizing `identity:get_project(project_id=63ed7377a21c45e6aa9864e1dfaafe56)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:14.600767 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c301e901-d232-4def-be7e-a36cf4a6d84b None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:14.602632 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 69/138] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 19 15:59:14 2022] GET /identity/v3/projects/63ed7377a21c45e6aa9864e1dfaafe56 => generated 312 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:14.609704 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c301e901-d232-4def-be7e-a36cf4a6d84b None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:14.730485 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c301e901-d232-4def-be7e-a36cf4a6d84b None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:14.776867 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe6534c-1283-4097-8998-3f59ee1e05c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:14.777959 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe6534c-1283-4097-8998-3f59ee1e05c8 None admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:14.777959 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe6534c-1283-4097-8998-3f59ee1e05c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:14.778110 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe6534c-1283-4097-8998-3f59ee1e05c8 None admin] PATH_INFO: `/v3/projects/63ed7377a21c45e6aa9864e1dfaafe56/users/bbf2a95469bc4f3593466124dfc85117/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:14.784507 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe6534c-1283-4097-8998-3f59ee1e05c8 None admin] RBAC: Authorizing `identity:create_grant(project_id=63ed7377a21c45e6aa9864e1dfaafe56, user_id=bbf2a95469bc4f3593466124dfc85117, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:14.785994 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe6534c-1283-4097-8998-3f59ee1e05c8 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:14.803532 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:14.813161 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4bb791-7c1b-46c9-80c1-4cbe8fae9661 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:14.814825 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 70/139] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Sep 19 15:59:14 2022] PUT /identity/v3/projects/63ed7377a21c45e6aa9864e1dfaafe56/users/bbf2a95469bc4f3593466124dfc85117/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 15:59:14.872433 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94c62d2-956c-48cc-b8c6-79935fe01a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:14.873130 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c62d2-956c-48cc-b8c6-79935fe01a53 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:14.873415 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c62d2-956c-48cc-b8c6-79935fe01a53 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:14.873699 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c62d2-956c-48cc-b8c6-79935fe01a53 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:14.875742 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94c62d2-956c-48cc-b8c6-79935fe01a53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:14.877381 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94c62d2-956c-48cc-b8c6-79935fe01a53 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:14.879288 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 70/140] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 15:59:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:14.885153 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe6534c-1283-4097-8998-3f59ee1e05c8 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:15.024491 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe6534c-1283-4097-8998-3f59ee1e05c8 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:15.076528 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f75e7-4600-4021-a4fd-2e8db0074a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:15.077240 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f75e7-4600-4021-a4fd-2e8db0074a20 None admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:15.077568 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f75e7-4600-4021-a4fd-2e8db0074a20 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:15.077864 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f75e7-4600-4021-a4fd-2e8db0074a20 None admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:15.078603 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153f75e7-4600-4021-a4fd-2e8db0074a20 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:15.080109 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153f75e7-4600-4021-a4fd-2e8db0074a20 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:15.116215 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 71/141] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Sep 19 15:59:14 2022] POST /identity/v3/users => generated 283 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 15:59:15.195736 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f8f37-ac47-4a1f-aa3f-49dc0c7e4f80 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:15.196074 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f8f37-ac47-4a1f-aa3f-49dc0c7e4f80 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:15.196313 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f8f37-ac47-4a1f-aa3f-49dc0c7e4f80 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:15.198237 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 71/142] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:59:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 15:59:15.208672 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69719441-cfe1-4b33-a900-ef5d158592f0 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:15.208672 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69719441-cfe1-4b33-a900-ef5d158592f0 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:15.208672 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69719441-cfe1-4b33-a900-ef5d158592f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:15.229543 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-69719441-cfe1-4b33-a900-ef5d158592f0 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:59:15.231578 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-69719441-cfe1-4b33-a900-ef5d158592f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:59:15.298557 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:15.321969 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 72/143] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:59:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 15:59:15.533333 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80652b89-fc1e-4d18-9838-89f538787798 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:15.533726 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80652b89-fc1e-4d18-9838-89f538787798 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:15.533924 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80652b89-fc1e-4d18-9838-89f538787798 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:15.535269 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 72/144] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:59:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 15:59:15.540659 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f854a83d-4dbb-4c37-b628-7f7eccf5c87a None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:15.541472 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f854a83d-4dbb-4c37-b628-7f7eccf5c87a None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:15.541946 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f854a83d-4dbb-4c37-b628-7f7eccf5c87a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:15.566969 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-f854a83d-4dbb-4c37-b628-7f7eccf5c87a None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:59:15.568479 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-f854a83d-4dbb-4c37-b628-7f7eccf5c87a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:59:15.631494 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80652b89-fc1e-4d18-9838-89f538787798 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:15.648608 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 73/145] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:59:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 15:59:15.654441 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:15.781555 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80652b89-fc1e-4d18-9838-89f538787798 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:15.827602 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431c1492-e7c6-4446-8ed3-2d5e8cd2786b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:15.829015 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431c1492-e7c6-4446-8ed3-2d5e8cd2786b None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:15.829508 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431c1492-e7c6-4446-8ed3-2d5e8cd2786b None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:15.829983 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431c1492-e7c6-4446-8ed3-2d5e8cd2786b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:15.841394 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431c1492-e7c6-4446-8ed3-2d5e8cd2786b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 15:59:15.847232 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-431c1492-e7c6-4446-8ed3-2d5e8cd2786b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 15:59:15.847232 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 15:59:15.847232 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 15:59:15.847232 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 15:59:15.847232 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 15:59:15.847232 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 15:59:15.847232 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 15:59:15.847232 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 15:59:15.847232 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 15:59:15.847232 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 15:59:15.847232 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 15:59:15.847232 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 15:59:15.847232 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 15:59:15.847232 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 15:59:15.847232 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 15:59:15.847232 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 15:59:15.848103 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 15:59:15.848103 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 15:59:15.848103 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 15:59:15.848103 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 15:59:15.848103 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 15:59:15.848103 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 15:59:15.848103 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 15:59:15.848103 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 15:59:15.848103 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 15:59:15.848103 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 15:59:15.848103 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 15:59:15.848103 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 15:59:15.848103 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 15:59:15.848103 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 15:59:15.848103 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 15:59:15.848103 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 15:59:15.848883 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 15:59:15.848883 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 15:59:15.848883 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 15:59:15.848883 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 15:59:15.848883 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 15:59:15.848883 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 15:59:15.848883 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 15:59:15.848883 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 15:59:15.848883 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 15:59:15.848883 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 15:59:15.848883 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 15:59:15.848883 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 15:59:15.851191 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 73/146] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 15:59:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 15:59:15.864614 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f854a83d-4dbb-4c37-b628-7f7eccf5c87a None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:16.013578 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f854a83d-4dbb-4c37-b628-7f7eccf5c87a None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:16.036105 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:16.057513 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34c6f85-7d0f-48b2-8812-077b7cbc9e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:16.058131 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c6f85-7d0f-48b2-8812-077b7cbc9e2e None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:16.058500 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c6f85-7d0f-48b2-8812-077b7cbc9e2e None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:16.058774 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c6f85-7d0f-48b2-8812-077b7cbc9e2e None admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:16.059445 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34c6f85-7d0f-48b2-8812-077b7cbc9e2e None admin] RBAC: Adding query filter params (name=member) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:16.059744 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34c6f85-7d0f-48b2-8812-077b7cbc9e2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:16.061120 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34c6f85-7d0f-48b2-8812-077b7cbc9e2e None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:16.065985 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431c1492-e7c6-4446-8ed3-2d5e8cd2786b None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:16.074316 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 74/147] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Sep 19 15:59:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:16.081089 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34c6f85-7d0f-48b2-8812-077b7cbc9e2e None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:16.226059 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:16.272070 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34c6f85-7d0f-48b2-8812-077b7cbc9e2e None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:16.289464 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431c1492-e7c6-4446-8ed3-2d5e8cd2786b None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:16.344082 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e5c404-593b-47ad-9670-a991a3cd66a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:16.344638 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e5c404-593b-47ad-9670-a991a3cd66a7 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:16.344912 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e5c404-593b-47ad-9670-a991a3cd66a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:16.345178 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e5c404-593b-47ad-9670-a991a3cd66a7 None admin] PATH_INFO: `/v3/users/bbf2a95469bc4f3593466124dfc85117` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:16.346511 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e5c404-593b-47ad-9670-a991a3cd66a7 None admin] RBAC: Authorizing `identity:get_user(user_id=bbf2a95469bc4f3593466124dfc85117)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:16.348903 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e5c404-593b-47ad-9670-a991a3cd66a7 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:16.350579 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 75/148] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 19 15:59:16 2022] GET /identity/v3/users/bbf2a95469bc4f3593466124dfc85117 => generated 275 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 15:59:16.356335 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e5c404-593b-47ad-9670-a991a3cd66a7 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:16.363873 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d559335-8128-444f-b30f-47dddc8e7934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:16.364522 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d559335-8128-444f-b30f-47dddc8e7934 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:16.364850 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d559335-8128-444f-b30f-47dddc8e7934 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:16.365167 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d559335-8128-444f-b30f-47dddc8e7934 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:16.386936 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d559335-8128-444f-b30f-47dddc8e7934 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 15:59:16.391831 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-4d559335-8128-444f-b30f-47dddc8e7934 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 15:59:16.391831 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 15:59:16.391831 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 15:59:16.391831 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 15:59:16.391831 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 15:59:16.391831 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 15:59:16.391831 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 15:59:16.391831 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 15:59:16.391831 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 15:59:16.391831 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 15:59:16.391831 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 15:59:16.391831 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 15:59:16.391831 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 15:59:16.391831 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 15:59:16.391831 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 15:59:16.391831 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 15:59:16.392660 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 15:59:16.392660 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 15:59:16.392660 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 15:59:16.392660 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 15:59:16.392660 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 15:59:16.392660 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 15:59:16.392660 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 15:59:16.392660 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 15:59:16.392660 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 15:59:16.392660 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 15:59:16.392660 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 15:59:16.392660 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 15:59:16.392660 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 15:59:16.392660 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 15:59:16.392660 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 15:59:16.392660 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 15:59:16.393414 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 15:59:16.393414 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 15:59:16.393414 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 15:59:16.393414 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 15:59:16.393414 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 15:59:16.393414 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 15:59:16.393414 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 15:59:16.393414 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 15:59:16.393414 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 15:59:16.393414 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 15:59:16.393414 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 15:59:16.393414 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 15:59:16.393971 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 74/149] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 15:59:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 15:59:16.405216 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d559335-8128-444f-b30f-47dddc8e7934 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:16.453490 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:16.563232 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e5c404-593b-47ad-9670-a991a3cd66a7 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:16.602834 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d559335-8128-444f-b30f-47dddc8e7934 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:16.619555 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c219744d-e24f-4ae4-bd65-913d04482cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:16.620183 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c219744d-e24f-4ae4-bd65-913d04482cf9 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:16.620470 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c219744d-e24f-4ae4-bd65-913d04482cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:16.620742 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c219744d-e24f-4ae4-bd65-913d04482cf9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:16.645899 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:16.657967 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e646ed-e4de-41ff-8cc3-df3c0eabd087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:16.659057 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e646ed-e4de-41ff-8cc3-df3c0eabd087 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:16.659547 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e646ed-e4de-41ff-8cc3-df3c0eabd087 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:16.660033 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e646ed-e4de-41ff-8cc3-df3c0eabd087 None admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:16.661276 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e646ed-e4de-41ff-8cc3-df3c0eabd087 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:16.661593 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e646ed-e4de-41ff-8cc3-df3c0eabd087 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:16.667138 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e646ed-e4de-41ff-8cc3-df3c0eabd087 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:16.684511 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 75/150] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Sep 19 15:59:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:16.691708 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e646ed-e4de-41ff-8cc3-df3c0eabd087 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:16.784520 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c219744d-e24f-4ae4-bd65-913d04482cf9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:16.786001 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c219744d-e24f-4ae4-bd65-913d04482cf9 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:16.828236 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e646ed-e4de-41ff-8cc3-df3c0eabd087 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:16.865332 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424c19b6-a623-45df-9b15-2dfd62c38c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:16.865893 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c19b6-a623-45df-9b15-2dfd62c38c5c None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:16.866171 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c19b6-a623-45df-9b15-2dfd62c38c5c None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:16.866474 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c19b6-a623-45df-9b15-2dfd62c38c5c None admin] PATH_INFO: `/v3/users/3a951e2fa39c4e7c8305f6e6067c55f4` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:16.867638 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424c19b6-a623-45df-9b15-2dfd62c38c5c None admin] RBAC: Authorizing `identity:get_user(user_id=3a951e2fa39c4e7c8305f6e6067c55f4)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:16.869219 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424c19b6-a623-45df-9b15-2dfd62c38c5c None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:16.870989 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 76/151] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 19 15:59:16 2022] GET /identity/v3/users/3a951e2fa39c4e7c8305f6e6067c55f4 => generated 289 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:16.875974 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:16.877183 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424c19b6-a623-45df-9b15-2dfd62c38c5c None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:16.954431 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 76/152] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 19 15:59:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 598 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 15:59:16.957495 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:16.960661 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c219744d-e24f-4ae4-bd65-913d04482cf9 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:16.995887 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424c19b6-a623-45df-9b15-2dfd62c38c5c None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:17.037588 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b1e0d0-1c7d-49fe-a48c-dd36d94378bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:17.038241 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b1e0d0-1c7d-49fe-a48c-dd36d94378bf None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:17.038736 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b1e0d0-1c7d-49fe-a48c-dd36d94378bf None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:17.039020 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b1e0d0-1c7d-49fe-a48c-dd36d94378bf None admin] PATH_INFO: `/v3/projects/ad7bddf539af4f8dabada63995dab07f` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:17.040429 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b1e0d0-1c7d-49fe-a48c-dd36d94378bf None admin] RBAC: Authorizing `identity:get_project(project_id=ad7bddf539af4f8dabada63995dab07f)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:17.042163 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b1e0d0-1c7d-49fe-a48c-dd36d94378bf None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:17.044145 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 77/153] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 19 15:59:16 2022] GET /identity/v3/projects/ad7bddf539af4f8dabada63995dab07f => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:17.050891 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b1e0d0-1c7d-49fe-a48c-dd36d94378bf None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:17.106613 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c219744d-e24f-4ae4-bd65-913d04482cf9 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:17.154992 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897862de-26a9-4311-8371-14f52c2c1939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:17.156098 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897862de-26a9-4311-8371-14f52c2c1939 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:17.156375 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897862de-26a9-4311-8371-14f52c2c1939 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:17.156647 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897862de-26a9-4311-8371-14f52c2c1939 None admin] PATH_INFO: `/v3/projects/ad7bddf539af4f8dabada63995dab07f` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:17.163176 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897862de-26a9-4311-8371-14f52c2c1939 None admin] RBAC: Authorizing `identity:get_project(project_id=ad7bddf539af4f8dabada63995dab07f)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:17.164649 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897862de-26a9-4311-8371-14f52c2c1939 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:17.165742 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b1e0d0-1c7d-49fe-a48c-dd36d94378bf None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:17.166652 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 77/154] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 19 15:59:16 2022] GET /identity/v3/projects/ad7bddf539af4f8dabada63995dab07f => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 15:59:17.172994 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897862de-26a9-4311-8371-14f52c2c1939 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:17.197289 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:17.201750 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:17.209720 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdfa06e-da62-42b5-a63e-02128884e2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:17.210539 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdfa06e-da62-42b5-a63e-02128884e2fb None admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:17.211225 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdfa06e-da62-42b5-a63e-02128884e2fb None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:17.211225 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdfa06e-da62-42b5-a63e-02128884e2fb None admin] PATH_INFO: `/v3/projects/ad7bddf539af4f8dabada63995dab07f/users/3a951e2fa39c4e7c8305f6e6067c55f4/roles/625d911d4d934c1e80f0a57620077d49` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:17.213422 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdfa06e-da62-42b5-a63e-02128884e2fb None admin] RBAC: Authorizing `identity:create_grant(project_id=ad7bddf539af4f8dabada63995dab07f, user_id=3a951e2fa39c4e7c8305f6e6067c55f4, role_id=625d911d4d934c1e80f0a57620077d49)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:17.215206 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdfa06e-da62-42b5-a63e-02128884e2fb None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:17.230068 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 78/155] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Sep 19 15:59:17 2022] PUT /identity/v3/projects/ad7bddf539af4f8dabada63995dab07f/users/3a951e2fa39c4e7c8305f6e6067c55f4/roles/625d911d4d934c1e80f0a57620077d49 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 15:59:17.231469 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069dd10b-a979-4a94-8cc4-ae40d226e968 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:17.231821 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069dd10b-a979-4a94-8cc4-ae40d226e968 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:17.232148 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069dd10b-a979-4a94-8cc4-ae40d226e968 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:17.234112 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 79/156] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:59:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 15:59:17.241339 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b6fc9-9d25-4a09-804f-c24ce2b85102 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:17.241639 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b6fc9-9d25-4a09-804f-c24ce2b85102 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:17.241999 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b6fc9-9d25-4a09-804f-c24ce2b85102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:17.273094 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-f17b6fc9-9d25-4a09-804f-c24ce2b85102 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:59:17.274355 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-f17b6fc9-9d25-4a09-804f-c24ce2b85102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:59:17.354691 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 80/157] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:59:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 15:59:17.378947 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897862de-26a9-4311-8371-14f52c2c1939 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:17.431295 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e15fdad-25ab-4d25-91cf-d581fed3c681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:17.431845 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e15fdad-25ab-4d25-91cf-d581fed3c681 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:17.432124 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e15fdad-25ab-4d25-91cf-d581fed3c681 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:17.432406 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e15fdad-25ab-4d25-91cf-d581fed3c681 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:17.433164 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e15fdad-25ab-4d25-91cf-d581fed3c681 None admin] RBAC: Adding query filter params (role.id=31b0628699ea4f98824f20e7568748cc, scope.project.id=ad7bddf539af4f8dabada63995dab07f, user.id=bbf2a95469bc4f3593466124dfc85117) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:17.433472 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e15fdad-25ab-4d25-91cf-d581fed3c681 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:17.435028 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e15fdad-25ab-4d25-91cf-d581fed3c681 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:17.442470 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 78/158] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 19 15:59:17 2022] GET /identity/v3/role_assignments?user.id=bbf2a95469bc4f3593466124dfc85117&scope.project.id=ad7bddf539af4f8dabada63995dab07f&role.id=31b0628699ea4f98824f20e7568748cc => generated 608 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 15:59:17.911920 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17b6fc9-9d25-4a09-804f-c24ce2b85102 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:18.050032 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17b6fc9-9d25-4a09-804f-c24ce2b85102 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:18.082792 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:18.092649 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2566853b-175f-4a50-bd95-787273b0028f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:18.093590 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2566853b-175f-4a50-bd95-787273b0028f None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:18.094387 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2566853b-175f-4a50-bd95-787273b0028f None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:18.098992 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2566853b-175f-4a50-bd95-787273b0028f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:18.111925 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2566853b-175f-4a50-bd95-787273b0028f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 15:59:18.116895 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-2566853b-175f-4a50-bd95-787273b0028f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 15:59:18.116895 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 15:59:18.116895 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 15:59:18.116895 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 15:59:18.116895 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 15:59:18.116895 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 15:59:18.116895 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 15:59:18.116895 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 15:59:18.116895 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 15:59:18.116895 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 15:59:18.116895 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 15:59:18.116895 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 15:59:18.116895 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 15:59:18.116895 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 15:59:18.116895 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 15:59:18.116895 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 15:59:18.117814 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 15:59:18.117814 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 15:59:18.117814 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 15:59:18.117814 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 15:59:18.117814 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 15:59:18.117814 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 15:59:18.117814 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 15:59:18.117814 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 15:59:18.117814 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 15:59:18.117814 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 15:59:18.117814 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 15:59:18.117814 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 15:59:18.117814 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 15:59:18.117814 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 15:59:18.117814 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 15:59:18.117814 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 15:59:18.118936 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 15:59:18.118936 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 15:59:18.118936 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 15:59:18.118936 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 15:59:18.118936 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 15:59:18.118936 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 15:59:18.118936 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 15:59:18.118936 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 15:59:18.118936 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 15:59:18.118936 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 15:59:18.118936 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 15:59:18.118936 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 15:59:18.118936 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 81/159] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 19 15:59:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 15:59:18.127464 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e15fdad-25ab-4d25-91cf-d581fed3c681 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:18.251515 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:18.279187 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e15fdad-25ab-4d25-91cf-d581fed3c681 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:18.321123 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a1e772-7aeb-4e5c-a1df-7df3f5c9fa65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:18.321663 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a1e772-7aeb-4e5c-a1df-7df3f5c9fa65 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:18.321943 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a1e772-7aeb-4e5c-a1df-7df3f5c9fa65 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:18.322213 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a1e772-7aeb-4e5c-a1df-7df3f5c9fa65 None admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:18.322917 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a1e772-7aeb-4e5c-a1df-7df3f5c9fa65 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:18.323223 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a1e772-7aeb-4e5c-a1df-7df3f5c9fa65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:18.324609 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a1e772-7aeb-4e5c-a1df-7df3f5c9fa65 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:18.338423 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 79/160] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 15:59:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:18.345105 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2566853b-175f-4a50-bd95-787273b0028f None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:18.521116 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2566853b-175f-4a50-bd95-787273b0028f None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:18.558613 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07742b61-5cf4-46c7-8908-6712e31a8394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:18.559249 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07742b61-5cf4-46c7-8908-6712e31a8394 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:18.559523 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07742b61-5cf4-46c7-8908-6712e31a8394 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:18.559791 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07742b61-5cf4-46c7-8908-6712e31a8394 None admin] PATH_INFO: `/v3/users/18d4b3afb42f46999e6cf7b500999690` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:18.560928 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07742b61-5cf4-46c7-8908-6712e31a8394 None admin] RBAC: Authorizing `identity:get_user(user_id=18d4b3afb42f46999e6cf7b500999690)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:18.562972 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07742b61-5cf4-46c7-8908-6712e31a8394 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:18.564929 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 82/161] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 19 15:59:18 2022] GET /identity/v3/users/18d4b3afb42f46999e6cf7b500999690 => generated 247 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:18.570548 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a1e772-7aeb-4e5c-a1df-7df3f5c9fa65 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:18.714407 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a1e772-7aeb-4e5c-a1df-7df3f5c9fa65 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:18.758161 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ebd20-8847-49dc-8e98-509abc329831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:18.758809 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ebd20-8847-49dc-8e98-509abc329831 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:18.759088 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ebd20-8847-49dc-8e98-509abc329831 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:18.759361 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ebd20-8847-49dc-8e98-509abc329831 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:18.776464 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:18.809773 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1111d598-93a3-40f7-923c-f55222ad5276 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:18.810428 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1111d598-93a3-40f7-923c-f55222ad5276 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:18.810913 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1111d598-93a3-40f7-923c-f55222ad5276 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:18.813061 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 83/162] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:59:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 15:59:18.818620 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ec0a6-2942-4da0-9177-628239e2f6a5 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:18.819164 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ec0a6-2942-4da0-9177-628239e2f6a5 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:18.819628 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ec0a6-2942-4da0-9177-628239e2f6a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:18.851506 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-836ec0a6-2942-4da0-9177-628239e2f6a5 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:59:18.853071 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-836ec0a6-2942-4da0-9177-628239e2f6a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:59:18.873993 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06ebd20-8847-49dc-8e98-509abc329831 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:18.876037 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06ebd20-8847-49dc-8e98-509abc329831 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:18.918586 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:18.961245 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 84/163] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:59:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 15:59:19.083102 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 80/164] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 19 15:59:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 15:59:19.084371 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb01e7-fe40-472d-8c40-bbf9b1c5bcb4 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:19.084664 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb01e7-fe40-472d-8c40-bbf9b1c5bcb4 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:19.084935 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb01e7-fe40-472d-8c40-bbf9b1c5bcb4 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:19.089296 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ebd20-8847-49dc-8e98-509abc329831 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:19.090854 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 85/165] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:59:19 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 15:59:19.096783 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec2ac76-d713-4731-9213-aa4259ad1f5f None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:19.097261 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec2ac76-d713-4731-9213-aa4259ad1f5f None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:19.097689 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec2ac76-d713-4731-9213-aa4259ad1f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:19.137902 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-1ec2ac76-d713-4731-9213-aa4259ad1f5f None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:59:19.139491 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-1ec2ac76-d713-4731-9213-aa4259ad1f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:59:19.158789 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:19.272123 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 86/166] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:59:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 15:59:19.291609 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ebd20-8847-49dc-8e98-509abc329831 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:19.343586 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae79e52-4fba-458e-896f-c08dcfc5b38e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:19.344137 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae79e52-4fba-458e-896f-c08dcfc5b38e None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:19.344415 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae79e52-4fba-458e-896f-c08dcfc5b38e None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:19.344683 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae79e52-4fba-458e-896f-c08dcfc5b38e None admin] PATH_INFO: `/v3/projects/ad7bddf539af4f8dabada63995dab07f` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:19.345839 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae79e52-4fba-458e-896f-c08dcfc5b38e None admin] RBAC: Authorizing `identity:get_project(project_id=ad7bddf539af4f8dabada63995dab07f)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:19.347309 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae79e52-4fba-458e-896f-c08dcfc5b38e None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:19.349322 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 81/167] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 19 15:59:19 2022] GET /identity/v3/projects/ad7bddf539af4f8dabada63995dab07f => generated 298 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 15:59:19.354987 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec2ac76-d713-4731-9213-aa4259ad1f5f None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:19.500126 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec2ac76-d713-4731-9213-aa4259ad1f5f None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:19.509854 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:19.537178 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4001e710-bc2e-487e-9ca2-cf525bcd7d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:19.537827 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4001e710-bc2e-487e-9ca2-cf525bcd7d0e None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:19.538096 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4001e710-bc2e-487e-9ca2-cf525bcd7d0e None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:19.538363 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4001e710-bc2e-487e-9ca2-cf525bcd7d0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:19.539101 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4001e710-bc2e-487e-9ca2-cf525bcd7d0e None admin] RBAC: Adding query filter params (role.id=63d2d197bfd6497583e94bfb065ce29b, scope.project.id=ad7bddf539af4f8dabada63995dab07f, user.id=18d4b3afb42f46999e6cf7b500999690) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:19.539403 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4001e710-bc2e-487e-9ca2-cf525bcd7d0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:19.540839 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4001e710-bc2e-487e-9ca2-cf525bcd7d0e None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:19.547141 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 87/168] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 19 15:59:19 2022] GET /identity/v3/role_assignments?user.id=18d4b3afb42f46999e6cf7b500999690&scope.project.id=ad7bddf539af4f8dabada63995dab07f&role.id=63d2d197bfd6497583e94bfb065ce29b => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:19.751387 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae79e52-4fba-458e-896f-c08dcfc5b38e None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:19.890975 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae79e52-4fba-458e-896f-c08dcfc5b38e None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:19.901037 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:19.931053 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1e6deb-ef6f-4306-87e2-50d7ffda5b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:19.931690 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e6deb-ef6f-4306-87e2-50d7ffda5b58 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:19.931966 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e6deb-ef6f-4306-87e2-50d7ffda5b58 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:19.932281 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e6deb-ef6f-4306-87e2-50d7ffda5b58 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:19.947562 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1e6deb-ef6f-4306-87e2-50d7ffda5b58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 15:59:19.953751 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-af1e6deb-ef6f-4306-87e2-50d7ffda5b58 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 15:59:19.953751 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 15:59:19.953751 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 15:59:19.953751 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 15:59:19.953751 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 15:59:19.953751 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 15:59:19.953751 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 15:59:19.953751 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 15:59:19.953751 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 15:59:19.953751 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 15:59:19.953751 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 15:59:19.953751 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 15:59:19.953751 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 15:59:19.953751 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 15:59:19.953751 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 15:59:19.953751 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 15:59:19.954671 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 15:59:19.954671 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 15:59:19.954671 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 15:59:19.954671 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 15:59:19.954671 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 15:59:19.954671 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 15:59:19.954671 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 15:59:19.954671 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 15:59:19.954671 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 15:59:19.954671 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 15:59:19.954671 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 15:59:19.954671 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 15:59:19.954671 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 15:59:19.954671 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 15:59:19.954671 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 15:59:19.954671 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 15:59:19.955791 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 15:59:19.955791 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 15:59:19.955791 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 15:59:19.955791 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 15:59:19.955791 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 15:59:19.955791 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 15:59:19.955791 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 15:59:19.955791 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 15:59:19.955791 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 15:59:19.955791 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 15:59:19.955791 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 15:59:19.955791 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 15:59:19.955791 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 82/169] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Sep 19 15:59:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 15:59:19.961882 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4001e710-bc2e-487e-9ca2-cf525bcd7d0e None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:19.966217 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1e6deb-ef6f-4306-87e2-50d7ffda5b58 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:20.123810 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4001e710-bc2e-487e-9ca2-cf525bcd7d0e None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:20.165112 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a381ab-fa4f-4e78-ada7-55c9efe44e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:20.165749 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a381ab-fa4f-4e78-ada7-55c9efe44e41 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:20.166277 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a381ab-fa4f-4e78-ada7-55c9efe44e41 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:20.166604 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a381ab-fa4f-4e78-ada7-55c9efe44e41 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:20.178327 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a381ab-fa4f-4e78-ada7-55c9efe44e41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 15:59:20.183958 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-39a381ab-fa4f-4e78-ada7-55c9efe44e41 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 15:59:20.183958 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 15:59:20.183958 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 15:59:20.183958 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 15:59:20.183958 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 15:59:20.183958 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 15:59:20.183958 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 15:59:20.183958 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 15:59:20.183958 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 15:59:20.183958 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 15:59:20.183958 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 15:59:20.183958 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 15:59:20.183958 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 15:59:20.183958 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 15:59:20.183958 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 15:59:20.183958 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 15:59:20.184817 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 15:59:20.184817 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 15:59:20.184817 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 15:59:20.184817 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 15:59:20.184817 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 15:59:20.184817 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 15:59:20.184817 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 15:59:20.184817 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 15:59:20.184817 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 15:59:20.184817 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 15:59:20.184817 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 15:59:20.184817 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 15:59:20.184817 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 15:59:20.184817 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 15:59:20.184817 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 15:59:20.184817 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 15:59:20.185983 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 15:59:20.185983 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 15:59:20.185983 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 15:59:20.185983 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 15:59:20.185983 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 15:59:20.185983 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 15:59:20.185983 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 15:59:20.185983 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 15:59:20.185983 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 15:59:20.185983 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 15:59:20.185983 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 15:59:20.185983 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 15:59:20.185983 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 88/170] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 15:59:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 15:59:20.187283 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3179cb-c8cc-400e-b76b-fc2ec0bce3cb None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:20.187283 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3179cb-c8cc-400e-b76b-fc2ec0bce3cb None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:20.187646 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3179cb-c8cc-400e-b76b-fc2ec0bce3cb None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:20.188895 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 89/171] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:59:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 15:59:20.194883 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd36377-3209-4dfa-a84d-9100a341ec3f None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:20.195169 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd36377-3209-4dfa-a84d-9100a341ec3f None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:20.195418 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd36377-3209-4dfa-a84d-9100a341ec3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:20.200735 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1e6deb-ef6f-4306-87e2-50d7ffda5b58 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:20.215116 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-0bd36377-3209-4dfa-a84d-9100a341ec3f None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:59:20.216441 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-0bd36377-3209-4dfa-a84d-9100a341ec3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:59:20.248494 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:20.287812 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9085f847-6645-4397-ab63-fdf2f0064aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:20.288513 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9085f847-6645-4397-ab63-fdf2f0064aa6 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:20.288797 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9085f847-6645-4397-ab63-fdf2f0064aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:20.289072 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9085f847-6645-4397-ab63-fdf2f0064aa6 None admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:20.289774 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9085f847-6645-4397-ab63-fdf2f0064aa6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:20.290140 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9085f847-6645-4397-ab63-fdf2f0064aa6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:20.291907 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9085f847-6645-4397-ab63-fdf2f0064aa6 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:20.305247 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 83/172] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 19 15:59:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:20.310305 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9085f847-6645-4397-ab63-fdf2f0064aa6 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:20.310733 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 90/173] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:59:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 15:59:20.312502 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd36377-3209-4dfa-a84d-9100a341ec3f None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:20.361568 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:20.439569 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9085f847-6645-4397-ab63-fdf2f0064aa6 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:20.495210 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de37d362-a6e9-4cea-b928-1ce3c6750b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:20.495851 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de37d362-a6e9-4cea-b928-1ce3c6750b75 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:20.496135 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de37d362-a6e9-4cea-b928-1ce3c6750b75 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:20.496408 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de37d362-a6e9-4cea-b928-1ce3c6750b75 None admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:20.497090 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de37d362-a6e9-4cea-b928-1ce3c6750b75 None admin] RBAC: Adding query filter params (name=member) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:20.497396 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de37d362-a6e9-4cea-b928-1ce3c6750b75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:20.498801 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de37d362-a6e9-4cea-b928-1ce3c6750b75 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:20.504540 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd36377-3209-4dfa-a84d-9100a341ec3f None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:20.513747 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 84/174] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Sep 19 15:59:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:20.519968 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de37d362-a6e9-4cea-b928-1ce3c6750b75 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:20.615134 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f17593-96e1-4129-a12e-ef2e6d77ea06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:20.615800 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f17593-96e1-4129-a12e-ef2e6d77ea06 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:20.616082 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f17593-96e1-4129-a12e-ef2e6d77ea06 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:20.616355 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f17593-96e1-4129-a12e-ef2e6d77ea06 None admin] PATH_INFO: `/v3/users/bbf2a95469bc4f3593466124dfc85117` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:20.617583 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f17593-96e1-4129-a12e-ef2e6d77ea06 None admin] RBAC: Authorizing `identity:get_user(user_id=bbf2a95469bc4f3593466124dfc85117)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:20.619078 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f17593-96e1-4129-a12e-ef2e6d77ea06 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:20.620883 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 91/175] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 19 15:59:20 2022] GET /identity/v3/users/bbf2a95469bc4f3593466124dfc85117 => generated 275 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:20.626224 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f17593-96e1-4129-a12e-ef2e6d77ea06 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:20.680404 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:20.703447 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de37d362-a6e9-4cea-b928-1ce3c6750b75 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:20.714574 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:20.749545 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203378bb-3dd4-461a-9777-0fd05a674c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:20.750395 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203378bb-3dd4-461a-9777-0fd05a674c70 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:20.750769 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203378bb-3dd4-461a-9777-0fd05a674c70 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:20.751139 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203378bb-3dd4-461a-9777-0fd05a674c70 None admin] PATH_INFO: `/v3/users/bbf2a95469bc4f3593466124dfc85117` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:20.752451 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203378bb-3dd4-461a-9777-0fd05a674c70 None admin] RBAC: Authorizing `identity:get_user(user_id=bbf2a95469bc4f3593466124dfc85117)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:20.753989 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203378bb-3dd4-461a-9777-0fd05a674c70 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:20.756040 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 85/176] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 19 15:59:20 2022] GET /identity/v3/users/bbf2a95469bc4f3593466124dfc85117 => generated 275 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 15:59:20.756944 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203378bb-3dd4-461a-9777-0fd05a674c70 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:20.857463 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f17593-96e1-4129-a12e-ef2e6d77ea06 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:20.880559 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203378bb-3dd4-461a-9777-0fd05a674c70 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:20.894343 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0a612c-7a1f-4c36-b6b4-cd2909122ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:20.894981 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0a612c-7a1f-4c36-b6b4-cd2909122ea0 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:20.895259 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0a612c-7a1f-4c36-b6b4-cd2909122ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:20.895533 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0a612c-7a1f-4c36-b6b4-cd2909122ea0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:20.918470 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a461594-ee2c-4be5-a50a-327967f93315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:20.919265 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a461594-ee2c-4be5-a50a-327967f93315 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:20.919569 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a461594-ee2c-4be5-a50a-327967f93315 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:20.919843 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a461594-ee2c-4be5-a50a-327967f93315 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:20.935115 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a461594-ee2c-4be5-a50a-327967f93315 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 15:59:20.941633 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-0a461594-ee2c-4be5-a50a-327967f93315 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 15:59:20.941633 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 15:59:20.941633 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 15:59:20.941633 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 15:59:20.941633 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 15:59:20.941633 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 15:59:20.941633 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 15:59:20.941633 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 15:59:20.941633 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 15:59:20.941633 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 15:59:20.941633 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 15:59:20.941633 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 15:59:20.941633 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 15:59:20.941633 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 15:59:20.941633 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 15:59:20.941633 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 15:59:20.942608 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 15:59:20.942608 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 15:59:20.942608 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 15:59:20.942608 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 15:59:20.942608 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 15:59:20.942608 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 15:59:20.942608 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 15:59:20.942608 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 15:59:20.942608 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 15:59:20.942608 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 15:59:20.942608 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 15:59:20.942608 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 15:59:20.942608 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 15:59:20.942608 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 15:59:20.942608 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 15:59:20.942608 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 15:59:20.943844 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 15:59:20.943844 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 15:59:20.943844 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 15:59:20.943844 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 15:59:20.943844 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 15:59:20.943844 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 15:59:20.943844 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 15:59:20.943844 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 15:59:20.943844 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 15:59:20.943844 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 15:59:20.943844 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 15:59:20.943844 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 15:59:20.943844 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 86/177] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 19 15:59:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 15:59:20.949617 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a461594-ee2c-4be5-a50a-327967f93315 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:20.976949 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:20.980731 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0a612c-7a1f-4c36-b6b4-cd2909122ea0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:20.982245 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0a612c-7a1f-4c36-b6b4-cd2909122ea0 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:21.016830 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:21.085942 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a461594-ee2c-4be5-a50a-327967f93315 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:21.099658 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 92/178] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 19 15:59:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 19 15:59:21.105602 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69638359-5738-4f99-bc9a-d36f9a4d50a3 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:21.105928 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69638359-5738-4f99-bc9a-d36f9a4d50a3 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:21.106178 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69638359-5738-4f99-bc9a-d36f9a4d50a3 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:21.107526 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 93/179] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:59:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 15:59:21.111348 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69638359-5738-4f99-bc9a-d36f9a4d50a3 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:21.132019 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5effa666-db8a-4ea6-aac7-cd68f093e18d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:21.132652 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5effa666-db8a-4ea6-aac7-cd68f093e18d None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:21.132928 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5effa666-db8a-4ea6-aac7-cd68f093e18d None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:21.133199 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5effa666-db8a-4ea6-aac7-cd68f093e18d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:21.206251 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5effa666-db8a-4ea6-aac7-cd68f093e18d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:21.208416 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5effa666-db8a-4ea6-aac7-cd68f093e18d None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:21.224073 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69638359-5738-4f99-bc9a-d36f9a4d50a3 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:21.232436 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:21.252521 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:21.258817 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2368b298-7254-4aa3-8785-030fa4b4d103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:21.259420 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2368b298-7254-4aa3-8785-030fa4b4d103 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:21.259695 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2368b298-7254-4aa3-8785-030fa4b4d103 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:21.259968 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2368b298-7254-4aa3-8785-030fa4b4d103 None admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:21.260631 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2368b298-7254-4aa3-8785-030fa4b4d103 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:21.260940 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2368b298-7254-4aa3-8785-030fa4b4d103 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:21.262321 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2368b298-7254-4aa3-8785-030fa4b4d103 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:21.271116 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 94/180] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 15:59:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:21.272270 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bfbf29-b56c-47ef-b7ca-27fb898cb053 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:21.272553 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bfbf29-b56c-47ef-b7ca-27fb898cb053 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:21.272854 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bfbf29-b56c-47ef-b7ca-27fb898cb053 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:21.274053 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 95/181] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:59:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 15:59:21.278493 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bfbf29-b56c-47ef-b7ca-27fb898cb053 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:21.320825 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 87/182] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 19 15:59:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 15:59:21.322074 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e26d4-7b6a-4733-936c-4e3b17402411 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:21.322353 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e26d4-7b6a-4733-936c-4e3b17402411 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:21.322653 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e26d4-7b6a-4733-936c-4e3b17402411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:21.340288 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-0e1e26d4-7b6a-4733-936c-4e3b17402411 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:59:21.341471 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-0e1e26d4-7b6a-4733-936c-4e3b17402411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:59:21.382811 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bfbf29-b56c-47ef-b7ca-27fb898cb053 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:21.406333 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 88/183] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:59:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 15:59:21.407596 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec96ae-9ed5-4b27-ab64-c89beb3f23ab None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:21.407885 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec96ae-9ed5-4b27-ab64-c89beb3f23ab None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:21.408172 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec96ae-9ed5-4b27-ab64-c89beb3f23ab None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:21.409699 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 89/184] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:59:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 15:59:21.410550 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ec96ae-9ed5-4b27-ab64-c89beb3f23ab None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:21.423616 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44c78dc-ac6d-4bd7-862b-58c25fe76ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:21.424646 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c78dc-ac6d-4bd7-862b-58c25fe76ec7 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:21.425136 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c78dc-ac6d-4bd7-862b-58c25fe76ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:21.425709 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c78dc-ac6d-4bd7-862b-58c25fe76ec7 None admin] PATH_INFO: `/v3/projects/ad7bddf539af4f8dabada63995dab07f` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:21.427113 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44c78dc-ac6d-4bd7-862b-58c25fe76ec7 None admin] RBAC: Authorizing `identity:get_project(project_id=ad7bddf539af4f8dabada63995dab07f)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:21.428822 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44c78dc-ac6d-4bd7-862b-58c25fe76ec7 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:21.431000 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 96/185] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 19 15:59:21 2022] GET /identity/v3/projects/ad7bddf539af4f8dabada63995dab07f => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:21.432384 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42654622-0206-471c-bbbc-88f85f0c68f6 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:21.432729 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42654622-0206-471c-bbbc-88f85f0c68f6 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:21.433080 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42654622-0206-471c-bbbc-88f85f0c68f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:21.451701 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-42654622-0206-471c-bbbc-88f85f0c68f6 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:59:21.453544 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-42654622-0206-471c-bbbc-88f85f0c68f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:59:21.474829 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:21.518636 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:21.528972 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ec96ae-9ed5-4b27-ab64-c89beb3f23ab None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:21.531439 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 97/186] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:59:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 15:59:21.532540 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42654622-0206-471c-bbbc-88f85f0c68f6 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:21.580961 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413ec7be-18d0-47fc-b67a-cff48d6c6b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:21.581883 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ec7be-18d0-47fc-b67a-cff48d6c6b7f None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:21.582358 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ec7be-18d0-47fc-b67a-cff48d6c6b7f None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:21.582831 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ec7be-18d0-47fc-b67a-cff48d6c6b7f None admin] PATH_INFO: `/v3/users/edf78c2cab38448f8c09f5c38ced338b` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:21.597129 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413ec7be-18d0-47fc-b67a-cff48d6c6b7f None admin] RBAC: Authorizing `identity:get_user(user_id=edf78c2cab38448f8c09f5c38ced338b)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:21.598927 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413ec7be-18d0-47fc-b67a-cff48d6c6b7f None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:21.600841 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 90/187] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 19 15:59:21 2022] GET /identity/v3/users/edf78c2cab38448f8c09f5c38ced338b => generated 283 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:21.606774 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c94fe8-6785-45e8-b717-24de071d6503 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:21.607255 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c94fe8-6785-45e8-b717-24de071d6503 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:21.607731 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c94fe8-6785-45e8-b717-24de071d6503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:21.626958 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-b9c94fe8-6785-45e8-b717-24de071d6503 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:59:21.628409 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-b9c94fe8-6785-45e8-b717-24de071d6503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:59:21.658457 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42654622-0206-471c-bbbc-88f85f0c68f6 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:21.695729 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16585098-586f-4be2-93f4-9f5ea345f39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:21.696366 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16585098-586f-4be2-93f4-9f5ea345f39c None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:21.696643 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16585098-586f-4be2-93f4-9f5ea345f39c None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:21.696913 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16585098-586f-4be2-93f4-9f5ea345f39c None admin] PATH_INFO: `/v3/projects/63ed7377a21c45e6aa9864e1dfaafe56` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:21.698019 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16585098-586f-4be2-93f4-9f5ea345f39c None admin] RBAC: Authorizing `identity:get_project(project_id=63ed7377a21c45e6aa9864e1dfaafe56)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:21.699221 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 91/188] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:59:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 15:59:21.700237 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c94fe8-6785-45e8-b717-24de071d6503 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:21.706661 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16585098-586f-4be2-93f4-9f5ea345f39c None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:21.708538 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 98/189] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 19 15:59:21 2022] GET /identity/v3/projects/63ed7377a21c45e6aa9864e1dfaafe56 => generated 312 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:21.709403 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16585098-586f-4be2-93f4-9f5ea345f39c None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:21.747303 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:21.792115 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:21.828853 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c94fe8-6785-45e8-b717-24de071d6503 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:21.842569 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16585098-586f-4be2-93f4-9f5ea345f39c None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:21.866672 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07db7d3-17c0-4caa-b11f-b10ac331c709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:21.867599 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07db7d3-17c0-4caa-b11f-b10ac331c709 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:21.868100 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07db7d3-17c0-4caa-b11f-b10ac331c709 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:21.868465 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07db7d3-17c0-4caa-b11f-b10ac331c709 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:21.869190 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07db7d3-17c0-4caa-b11f-b10ac331c709 None admin] RBAC: Adding query filter params (role.id=a1606b79a2c840238de6860bc69b49b7, scope.project.id=ad7bddf539af4f8dabada63995dab07f, user.id=bbf2a95469bc4f3593466124dfc85117) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:21.869495 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07db7d3-17c0-4caa-b11f-b10ac331c709 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:21.870949 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07db7d3-17c0-4caa-b11f-b10ac331c709 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:21.877824 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 92/190] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 19 15:59:21 2022] GET /identity/v3/role_assignments?user.id=bbf2a95469bc4f3593466124dfc85117&scope.project.id=ad7bddf539af4f8dabada63995dab07f&role.id=a1606b79a2c840238de6860bc69b49b7 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:21.879398 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07db7d3-17c0-4caa-b11f-b10ac331c709 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:21.887407 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de01ab78-595e-4b28-96a9-70d56102881d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:21.888417 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de01ab78-595e-4b28-96a9-70d56102881d None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:21.888920 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de01ab78-595e-4b28-96a9-70d56102881d None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:21.889658 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de01ab78-595e-4b28-96a9-70d56102881d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:21.978834 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de01ab78-595e-4b28-96a9-70d56102881d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:21.980555 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de01ab78-595e-4b28-96a9-70d56102881d None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:22.028947 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07db7d3-17c0-4caa-b11f-b10ac331c709 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:22.031392 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:22.057795 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:22.074074 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9ea732-786e-421c-91fb-9cd5ff65baa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:22.075060 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ea732-786e-421c-91fb-9cd5ff65baa5 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:22.075060 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ea732-786e-421c-91fb-9cd5ff65baa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:22.075341 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ea732-786e-421c-91fb-9cd5ff65baa5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:22.088349 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9ea732-786e-421c-91fb-9cd5ff65baa5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 15:59:22.093763 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-3e9ea732-786e-421c-91fb-9cd5ff65baa5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 15:59:22.093763 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 15:59:22.093763 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 15:59:22.093763 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 15:59:22.093763 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 15:59:22.093763 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 15:59:22.093763 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 15:59:22.093763 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 15:59:22.093763 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 15:59:22.093763 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 15:59:22.093763 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 15:59:22.093763 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 15:59:22.093763 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 15:59:22.093763 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 15:59:22.093763 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 15:59:22.093763 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 15:59:22.094784 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 15:59:22.094784 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 15:59:22.094784 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 15:59:22.094784 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 15:59:22.094784 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 15:59:22.094784 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 15:59:22.094784 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 15:59:22.094784 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 15:59:22.094784 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 15:59:22.094784 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 15:59:22.094784 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 15:59:22.094784 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 15:59:22.094784 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 15:59:22.094784 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 15:59:22.094784 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 15:59:22.094784 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 15:59:22.095530 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 15:59:22.095530 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 15:59:22.095530 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 15:59:22.095530 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 15:59:22.095530 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 15:59:22.095530 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 15:59:22.095530 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 15:59:22.095530 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 15:59:22.095530 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 15:59:22.095530 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 15:59:22.095530 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 15:59:22.095530 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 15:59:22.095530 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 93/191] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Sep 19 15:59:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 15:59:22.096174 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9ea732-786e-421c-91fb-9cd5ff65baa5 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:22.114288 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 99/192] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 19 15:59:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 15:59:22.117443 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de01ab78-595e-4b28-96a9-70d56102881d None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:22.202195 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9ea732-786e-421c-91fb-9cd5ff65baa5 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:22.223487 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de01ab78-595e-4b28-96a9-70d56102881d None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:22.235337 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26879e1a-9d82-47b2-b96e-56a9827df3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:22.235931 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26879e1a-9d82-47b2-b96e-56a9827df3a6 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:22.236231 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26879e1a-9d82-47b2-b96e-56a9827df3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:22.236500 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26879e1a-9d82-47b2-b96e-56a9827df3a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:22.237206 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26879e1a-9d82-47b2-b96e-56a9827df3a6 None admin] RBAC: Adding query filter params (role.id=31b0628699ea4f98824f20e7568748cc, scope.project.id=63ed7377a21c45e6aa9864e1dfaafe56, user.id=bbf2a95469bc4f3593466124dfc85117) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:22.237502 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26879e1a-9d82-47b2-b96e-56a9827df3a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:22.238977 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26879e1a-9d82-47b2-b96e-56a9827df3a6 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:22.244521 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 94/193] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 19 15:59:22 2022] GET /identity/v3/role_assignments?user.id=bbf2a95469bc4f3593466124dfc85117&scope.project.id=63ed7377a21c45e6aa9864e1dfaafe56&role.id=31b0628699ea4f98824f20e7568748cc => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:22.245296 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26879e1a-9d82-47b2-b96e-56a9827df3a6 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:22.262815 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edca4a0e-2763-4c8f-bd1a-552078b8d68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:22.263489 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca4a0e-2763-4c8f-bd1a-552078b8d68d None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:22.263806 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca4a0e-2763-4c8f-bd1a-552078b8d68d None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:22.264088 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca4a0e-2763-4c8f-bd1a-552078b8d68d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:22.265544 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edca4a0e-2763-4c8f-bd1a-552078b8d68d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:22.267414 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edca4a0e-2763-4c8f-bd1a-552078b8d68d None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:22.271039 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 100/194] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 15:59:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:22.271039 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edca4a0e-2763-4c8f-bd1a-552078b8d68d None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:22.278136 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:22.310131 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:22.373375 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26879e1a-9d82-47b2-b96e-56a9827df3a6 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:22.411180 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580da55f-4448-4e80-b6f3-db8692f55563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:22.411825 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580da55f-4448-4e80-b6f3-db8692f55563 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:22.412104 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580da55f-4448-4e80-b6f3-db8692f55563 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:22.412377 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580da55f-4448-4e80-b6f3-db8692f55563 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:22.415567 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edca4a0e-2763-4c8f-bd1a-552078b8d68d None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:22.424437 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580da55f-4448-4e80-b6f3-db8692f55563 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 15:59:22.430111 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-580da55f-4448-4e80-b6f3-db8692f55563 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 15:59:22.430111 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 15:59:22.430111 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 15:59:22.430111 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 15:59:22.430111 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 15:59:22.430111 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 15:59:22.430111 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 15:59:22.430111 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 15:59:22.430111 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 15:59:22.430111 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 15:59:22.430111 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 15:59:22.430111 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 15:59:22.430111 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 15:59:22.430111 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 15:59:22.430111 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 15:59:22.430111 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 15:59:22.431008 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 15:59:22.431008 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 15:59:22.431008 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 15:59:22.431008 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 15:59:22.431008 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 15:59:22.431008 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 15:59:22.431008 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 15:59:22.431008 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 15:59:22.431008 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 15:59:22.431008 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 15:59:22.431008 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 15:59:22.431008 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 15:59:22.431008 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 15:59:22.431008 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 15:59:22.431008 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 15:59:22.431008 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 15:59:22.431768 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 15:59:22.431768 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 15:59:22.431768 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 15:59:22.431768 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 15:59:22.431768 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 15:59:22.431768 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 15:59:22.431768 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 15:59:22.431768 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 15:59:22.431768 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 15:59:22.431768 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 15:59:22.431768 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 15:59:22.431768 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 15:59:22.431768 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 95/195] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 15:59:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 15:59:22.435484 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580da55f-4448-4e80-b6f3-db8692f55563 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:22.464896 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c84b551-92e6-47db-ae7a-3a16a9a08e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:22.465511 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c84b551-92e6-47db-ae7a-3a16a9a08e6d None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:22.465786 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c84b551-92e6-47db-ae7a-3a16a9a08e6d None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:22.466060 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c84b551-92e6-47db-ae7a-3a16a9a08e6d None admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:22.468229 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c84b551-92e6-47db-ae7a-3a16a9a08e6d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:22.468596 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c84b551-92e6-47db-ae7a-3a16a9a08e6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:22.469977 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c84b551-92e6-47db-ae7a-3a16a9a08e6d None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:22.480169 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 101/196] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 19 15:59:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:22.480980 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c84b551-92e6-47db-ae7a-3a16a9a08e6d None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:22.535541 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:22.553222 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580da55f-4448-4e80-b6f3-db8692f55563 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:22.572484 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:22.584512 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c84b551-92e6-47db-ae7a-3a16a9a08e6d None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:22.587706 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e5c929-bcb3-4355-8e31-c85834a305b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:22.587706 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e5c929-bcb3-4355-8e31-c85834a305b4 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:22.587706 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e5c929-bcb3-4355-8e31-c85834a305b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:22.588010 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e5c929-bcb3-4355-8e31-c85834a305b4 None admin] PATH_INFO: `/v3/projects/93b0db50dd5e4d3195beded37dd4105f` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:22.589975 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e5c929-bcb3-4355-8e31-c85834a305b4 None admin] RBAC: Authorizing `identity:get_project(project_id=93b0db50dd5e4d3195beded37dd4105f)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:22.591470 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e5c929-bcb3-4355-8e31-c85834a305b4 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:22.593613 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 96/197] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 19 15:59:22 2022] GET /identity/v3/projects/93b0db50dd5e4d3195beded37dd4105f => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:22.599457 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e5c929-bcb3-4355-8e31-c85834a305b4 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:22.620770 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e40f486-84ee-4e22-844f-2e530971fa10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:22.621377 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e40f486-84ee-4e22-844f-2e530971fa10 None admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:22.621657 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e40f486-84ee-4e22-844f-2e530971fa10 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:22.621924 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e40f486-84ee-4e22-844f-2e530971fa10 None admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:22.622601 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e40f486-84ee-4e22-844f-2e530971fa10 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:22.624081 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e40f486-84ee-4e22-844f-2e530971fa10 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:22.645192 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 102/198] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Sep 19 15:59:22 2022] POST /identity/v3/users => generated 297 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 15:59:22.651565 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e40f486-84ee-4e22-844f-2e530971fa10 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:22.710024 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e5c929-bcb3-4355-8e31-c85834a305b4 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:22.750797 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d6f872-b70f-4c20-90dc-ffba0ce109cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:22.751500 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d6f872-b70f-4c20-90dc-ffba0ce109cd None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:22.751781 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d6f872-b70f-4c20-90dc-ffba0ce109cd None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:22.752056 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d6f872-b70f-4c20-90dc-ffba0ce109cd None admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:22.752736 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d6f872-b70f-4c20-90dc-ffba0ce109cd None admin] RBAC: Adding query filter params (name=reader) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:22.753041 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d6f872-b70f-4c20-90dc-ffba0ce109cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:22.754865 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d6f872-b70f-4c20-90dc-ffba0ce109cd None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:22.765583 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 97/199] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Sep 19 15:59:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:22.771459 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d6f872-b70f-4c20-90dc-ffba0ce109cd None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:22.771638 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e40f486-84ee-4e22-844f-2e530971fa10 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:22.795530 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:22.812855 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5442092b-a271-4b20-8aef-e36ecd79becf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:22.813485 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5442092b-a271-4b20-8aef-e36ecd79becf None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:22.813765 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5442092b-a271-4b20-8aef-e36ecd79becf None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:22.814043 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5442092b-a271-4b20-8aef-e36ecd79becf None admin] PATH_INFO: `/v3/users/edf78c2cab38448f8c09f5c38ced338b` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:22.815255 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5442092b-a271-4b20-8aef-e36ecd79becf None admin] RBAC: Authorizing `identity:get_user(user_id=edf78c2cab38448f8c09f5c38ced338b)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:22.816713 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5442092b-a271-4b20-8aef-e36ecd79becf None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:22.818319 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 103/200] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 19 15:59:22 2022] GET /identity/v3/users/edf78c2cab38448f8c09f5c38ced338b => generated 283 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:22.824283 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5442092b-a271-4b20-8aef-e36ecd79becf None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:22.844912 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:22.904147 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d6f872-b70f-4c20-90dc-ffba0ce109cd None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:22.949258 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5442092b-a271-4b20-8aef-e36ecd79becf None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:22.955125 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f20b3ec-7581-4042-b7f9-7f8d20cbf4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:22.955857 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f20b3ec-7581-4042-b7f9-7f8d20cbf4dd None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:22.956212 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f20b3ec-7581-4042-b7f9-7f8d20cbf4dd None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:22.956558 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f20b3ec-7581-4042-b7f9-7f8d20cbf4dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:22.957376 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f20b3ec-7581-4042-b7f9-7f8d20cbf4dd None admin] RBAC: Adding query filter params (role.id=63d2d197bfd6497583e94bfb065ce29b, scope.project.id=93b0db50dd5e4d3195beded37dd4105f, user.id=edf78c2cab38448f8c09f5c38ced338b) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:22.957766 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f20b3ec-7581-4042-b7f9-7f8d20cbf4dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:22.959366 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f20b3ec-7581-4042-b7f9-7f8d20cbf4dd None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:22.967490 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 98/201] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 19 15:59:22 2022] GET /identity/v3/role_assignments?user.id=edf78c2cab38448f8c09f5c38ced338b&scope.project.id=93b0db50dd5e4d3195beded37dd4105f&role.id=63d2d197bfd6497583e94bfb065ce29b => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:22.968804 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f20b3ec-7581-4042-b7f9-7f8d20cbf4dd None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:23.005118 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419ef493-1ec8-438b-86ea-6e6f4fb3aad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:23.006681 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419ef493-1ec8-438b-86ea-6e6f4fb3aad5 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:23.006965 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419ef493-1ec8-438b-86ea-6e6f4fb3aad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:23.007552 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419ef493-1ec8-438b-86ea-6e6f4fb3aad5 None admin] PATH_INFO: `/v3/users/3a951e2fa39c4e7c8305f6e6067c55f4` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:23.009291 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419ef493-1ec8-438b-86ea-6e6f4fb3aad5 None admin] RBAC: Authorizing `identity:get_user(user_id=3a951e2fa39c4e7c8305f6e6067c55f4)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:23.011766 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419ef493-1ec8-438b-86ea-6e6f4fb3aad5 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:23.014542 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 104/202] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 19 15:59:22 2022] GET /identity/v3/users/3a951e2fa39c4e7c8305f6e6067c55f4 => generated 289 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:23.020669 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419ef493-1ec8-438b-86ea-6e6f4fb3aad5 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:23.087930 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:23.112237 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f20b3ec-7581-4042-b7f9-7f8d20cbf4dd None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:23.150011 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:23.159169 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4181974-5b45-4bb9-9b67-29ad8b5ae4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:23.159799 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4181974-5b45-4bb9-9b67-29ad8b5ae4ff None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:23.160076 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4181974-5b45-4bb9-9b67-29ad8b5ae4ff None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:23.160373 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4181974-5b45-4bb9-9b67-29ad8b5ae4ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:23.182747 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419ef493-1ec8-438b-86ea-6e6f4fb3aad5 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:23.226721 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea245ae8-4b88-4eda-9eb9-2d4b2087697e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:23.228163 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea245ae8-4b88-4eda-9eb9-2d4b2087697e None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:23.228860 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea245ae8-4b88-4eda-9eb9-2d4b2087697e None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:23.229362 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea245ae8-4b88-4eda-9eb9-2d4b2087697e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:23.249970 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4181974-5b45-4bb9-9b67-29ad8b5ae4ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:23.253756 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4181974-5b45-4bb9-9b67-29ad8b5ae4ff None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:23.329228 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea245ae8-4b88-4eda-9eb9-2d4b2087697e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:23.331571 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea245ae8-4b88-4eda-9eb9-2d4b2087697e None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:23.361186 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:23.415933 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 99/203] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 19 15:59:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 15:59:23.430582 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4181974-5b45-4bb9-9b67-29ad8b5ae4ff None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:23.440439 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:23.473439 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 105/204] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 19 15:59:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 19 15:59:23.479824 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea245ae8-4b88-4eda-9eb9-2d4b2087697e None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:23.592908 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea245ae8-4b88-4eda-9eb9-2d4b2087697e None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:23.637535 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687abce7-0435-40be-8430-62d65767a63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:23.638742 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687abce7-0435-40be-8430-62d65767a63f None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:23.639215 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687abce7-0435-40be-8430-62d65767a63f None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:23.639710 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687abce7-0435-40be-8430-62d65767a63f None admin] PATH_INFO: `/v3/projects/ad7bddf539af4f8dabada63995dab07f` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:23.641640 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687abce7-0435-40be-8430-62d65767a63f None admin] RBAC: Authorizing `identity:get_project(project_id=ad7bddf539af4f8dabada63995dab07f)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:23.643361 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687abce7-0435-40be-8430-62d65767a63f None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:23.645626 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 106/205] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 19 15:59:23 2022] GET /identity/v3/projects/ad7bddf539af4f8dabada63995dab07f => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:23.652451 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687abce7-0435-40be-8430-62d65767a63f None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:23.704635 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4181974-5b45-4bb9-9b67-29ad8b5ae4ff None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:23.717138 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:23.785434 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687abce7-0435-40be-8430-62d65767a63f None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:23.811789 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91515cd5-f607-46ad-9793-d467dde7c599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:23.812442 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91515cd5-f607-46ad-9793-d467dde7c599 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:23.812723 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91515cd5-f607-46ad-9793-d467dde7c599 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:23.812999 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91515cd5-f607-46ad-9793-d467dde7c599 None admin] PATH_INFO: `/v3/projects/93b0db50dd5e4d3195beded37dd4105f` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:23.816269 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91515cd5-f607-46ad-9793-d467dde7c599 None admin] RBAC: Authorizing `identity:get_project(project_id=93b0db50dd5e4d3195beded37dd4105f)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:23.817770 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91515cd5-f607-46ad-9793-d467dde7c599 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:23.824475 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3a7e7d-4990-46a1-a6c1-ea4c964c52eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:23.824671 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 100/206] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 19 15:59:23 2022] GET /identity/v3/projects/93b0db50dd5e4d3195beded37dd4105f => generated 302 bytes in 397 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:23.825495 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3a7e7d-4990-46a1-a6c1-ea4c964c52eb None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:23.825964 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3a7e7d-4990-46a1-a6c1-ea4c964c52eb None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:23.826493 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3a7e7d-4990-46a1-a6c1-ea4c964c52eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:23.827446 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3a7e7d-4990-46a1-a6c1-ea4c964c52eb None admin] RBAC: Adding query filter params (role.id=625d911d4d934c1e80f0a57620077d49, scope.project.id=ad7bddf539af4f8dabada63995dab07f, user.id=3a951e2fa39c4e7c8305f6e6067c55f4) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:23.827963 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3a7e7d-4990-46a1-a6c1-ea4c964c52eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:23.829609 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3a7e7d-4990-46a1-a6c1-ea4c964c52eb None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:23.831396 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91515cd5-f607-46ad-9793-d467dde7c599 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:23.838265 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 107/207] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 19 15:59:23 2022] GET /identity/v3/role_assignments?user.id=3a951e2fa39c4e7c8305f6e6067c55f4&scope.project.id=ad7bddf539af4f8dabada63995dab07f&role.id=625d911d4d934c1e80f0a57620077d49 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:23.915254 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:24.004282 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91515cd5-f607-46ad-9793-d467dde7c599 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:24.056610 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53a7c51-fb90-4a05-ab90-e542f494d4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:24.057604 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53a7c51-fb90-4a05-ab90-e542f494d4af None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:24.058082 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53a7c51-fb90-4a05-ab90-e542f494d4af None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:24.062795 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53a7c51-fb90-4a05-ab90-e542f494d4af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:24.064114 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53a7c51-fb90-4a05-ab90-e542f494d4af None admin] RBAC: Adding query filter params (role.id=a1606b79a2c840238de6860bc69b49b7, scope.project.id=93b0db50dd5e4d3195beded37dd4105f, user.id=edf78c2cab38448f8c09f5c38ced338b) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:24.064690 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53a7c51-fb90-4a05-ab90-e542f494d4af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:24.066583 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53a7c51-fb90-4a05-ab90-e542f494d4af None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:24.073661 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 101/208] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 19 15:59:23 2022] GET /identity/v3/role_assignments?user.id=edf78c2cab38448f8c09f5c38ced338b&scope.project.id=93b0db50dd5e4d3195beded37dd4105f&role.id=a1606b79a2c840238de6860bc69b49b7 => generated 259 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:25.388695 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a5b1c2-7af2-4f4f-b403-250a63a7b070 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:25.389245 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a5b1c2-7af2-4f4f-b403-250a63a7b070 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:25.389245 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a5b1c2-7af2-4f4f-b403-250a63a7b070 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:25.391267 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 108/209] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:59:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 15:59:25.396915 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458c35dd-7d51-45b9-94d5-32b045a4700c None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:25.397761 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458c35dd-7d51-45b9-94d5-32b045a4700c None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:25.398279 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458c35dd-7d51-45b9-94d5-32b045a4700c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:25.417651 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-458c35dd-7d51-45b9-94d5-32b045a4700c None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:59:25.418899 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-458c35dd-7d51-45b9-94d5-32b045a4700c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:59:25.486071 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef0068a-5c5e-4cf3-978a-7841dd5b69b1 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:25.486714 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef0068a-5c5e-4cf3-978a-7841dd5b69b1 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:25.487220 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef0068a-5c5e-4cf3-978a-7841dd5b69b1 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:25.489148 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 109/210] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:59:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 15:59:25.490359 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 102/211] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:59:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 15:59:25.494867 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30c595-5cbe-4b36-91fc-24ecf39f13d1 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:25.495205 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30c595-5cbe-4b36-91fc-24ecf39f13d1 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:25.495413 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30c595-5cbe-4b36-91fc-24ecf39f13d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:25.516271 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-ea30c595-5cbe-4b36-91fc-24ecf39f13d1 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:59:25.517508 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-ea30c595-5cbe-4b36-91fc-24ecf39f13d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:59:25.589438 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 110/212] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:59:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 15:59:25.631134 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68457e3a-d3f9-4a1a-ad76-e79f05824652 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:25.631448 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68457e3a-d3f9-4a1a-ad76-e79f05824652 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:25.631680 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68457e3a-d3f9-4a1a-ad76-e79f05824652 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:25.633025 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 103/213] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:59:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 15:59:25.638184 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380dd50-670d-48fc-b69f-802b4e2a811a None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:25.638502 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380dd50-670d-48fc-b69f-802b4e2a811a None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:25.638825 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380dd50-670d-48fc-b69f-802b4e2a811a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:25.657488 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-4380dd50-670d-48fc-b69f-802b4e2a811a None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:59:25.659009 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-4380dd50-670d-48fc-b69f-802b4e2a811a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:59:25.695998 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:25.729602 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 111/214] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:59:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 15:59:25.787821 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68457e3a-d3f9-4a1a-ad76-e79f05824652 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:25.834189 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:25.871763 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4380dd50-670d-48fc-b69f-802b4e2a811a None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:25.904133 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68457e3a-d3f9-4a1a-ad76-e79f05824652 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:25.946349 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8f0440-8f6e-43ea-91ec-1103fd9aec75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:25.947707 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8f0440-8f6e-43ea-91ec-1103fd9aec75 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:25.948422 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8f0440-8f6e-43ea-91ec-1103fd9aec75 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:25.948865 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8f0440-8f6e-43ea-91ec-1103fd9aec75 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:25.966887 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8f0440-8f6e-43ea-91ec-1103fd9aec75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 15:59:25.972606 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-5b8f0440-8f6e-43ea-91ec-1103fd9aec75 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 15:59:25.972606 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 15:59:25.972606 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 15:59:25.972606 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 15:59:25.972606 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 15:59:25.972606 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 15:59:25.972606 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 15:59:25.972606 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 15:59:25.972606 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 15:59:25.972606 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 15:59:25.972606 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 15:59:25.972606 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 15:59:25.972606 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 15:59:25.972606 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 15:59:25.972606 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 15:59:25.972606 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 15:59:25.973475 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 15:59:25.973475 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 15:59:25.973475 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 15:59:25.973475 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 15:59:25.973475 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 15:59:25.973475 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 15:59:25.973475 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 15:59:25.973475 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 15:59:25.973475 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 15:59:25.973475 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 15:59:25.973475 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 15:59:25.973475 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 15:59:25.973475 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 15:59:25.973475 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 15:59:25.973475 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 15:59:25.973475 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 15:59:25.974247 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 15:59:25.974247 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 15:59:25.974247 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 15:59:25.974247 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 15:59:25.974247 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 15:59:25.974247 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 15:59:25.974247 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 15:59:25.974247 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 15:59:25.974247 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 15:59:25.974247 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 15:59:25.974247 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 15:59:25.974247 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 15:59:25.977186 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 104/215] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 15:59:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 15:59:25.987562 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8f0440-8f6e-43ea-91ec-1103fd9aec75 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:26.000609 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4380dd50-670d-48fc-b69f-802b4e2a811a None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:26.046244 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8b404a-e070-4a5d-85d8-512b35e53732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:26.046926 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8b404a-e070-4a5d-85d8-512b35e53732 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:26.047209 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8b404a-e070-4a5d-85d8-512b35e53732 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:26.047484 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8b404a-e070-4a5d-85d8-512b35e53732 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:26.059444 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8b404a-e070-4a5d-85d8-512b35e53732 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 15:59:26.064879 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-7e8b404a-e070-4a5d-85d8-512b35e53732 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 15:59:26.064879 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 15:59:26.064879 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 15:59:26.064879 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 15:59:26.064879 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 15:59:26.064879 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 15:59:26.064879 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 15:59:26.064879 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 15:59:26.064879 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 15:59:26.064879 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 15:59:26.064879 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 15:59:26.064879 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 15:59:26.064879 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 15:59:26.064879 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 15:59:26.064879 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 15:59:26.064879 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 15:59:26.065959 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 15:59:26.065959 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 15:59:26.065959 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 15:59:26.065959 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 15:59:26.065959 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 15:59:26.065959 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 15:59:26.065959 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 15:59:26.065959 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 15:59:26.065959 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 15:59:26.065959 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 15:59:26.065959 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 15:59:26.065959 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 15:59:26.065959 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 15:59:26.065959 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 15:59:26.065959 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 15:59:26.065959 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 15:59:26.067200 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 15:59:26.067200 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 15:59:26.067200 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 15:59:26.067200 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 15:59:26.067200 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 15:59:26.067200 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 15:59:26.067200 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 15:59:26.067200 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 15:59:26.067200 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 15:59:26.067200 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 15:59:26.067200 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 15:59:26.067200 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 15:59:26.067200 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 112/216] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 19 15:59:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 15:59:26.067200 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8b404a-e070-4a5d-85d8-512b35e53732 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:26.105502 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:26.117274 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8f0440-8f6e-43ea-91ec-1103fd9aec75 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:26.118976 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:26.160648 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b52994-c19e-48f8-af91-01e0338df5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:26.161327 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b52994-c19e-48f8-af91-01e0338df5da None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:26.162011 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b52994-c19e-48f8-af91-01e0338df5da None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:26.162315 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b52994-c19e-48f8-af91-01e0338df5da None admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:26.164071 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b52994-c19e-48f8-af91-01e0338df5da None admin] RBAC: Adding query filter params (name=member) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:26.164998 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b52994-c19e-48f8-af91-01e0338df5da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:26.167545 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b52994-c19e-48f8-af91-01e0338df5da None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:26.179933 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 105/217] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Sep 19 15:59:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:26.181235 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b52994-c19e-48f8-af91-01e0338df5da None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:26.192002 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8b404a-e070-4a5d-85d8-512b35e53732 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:26.231672 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010c343c-7fe4-41ec-81d2-11512636cd3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:26.232358 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c343c-7fe4-41ec-81d2-11512636cd3f None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:26.232635 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c343c-7fe4-41ec-81d2-11512636cd3f None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:26.232905 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c343c-7fe4-41ec-81d2-11512636cd3f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:26.234095 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010c343c-7fe4-41ec-81d2-11512636cd3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:26.235584 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010c343c-7fe4-41ec-81d2-11512636cd3f None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:26.237385 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 113/218] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 15:59:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:26.237961 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010c343c-7fe4-41ec-81d2-11512636cd3f None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:26.299527 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b52994-c19e-48f8-af91-01e0338df5da None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:26.338844 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1f57db-0b7e-4c3c-a8d8-5bf3005ba1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:26.339745 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f57db-0b7e-4c3c-a8d8-5bf3005ba1f9 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:26.340300 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f57db-0b7e-4c3c-a8d8-5bf3005ba1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:26.340846 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f57db-0b7e-4c3c-a8d8-5bf3005ba1f9 None admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:26.341675 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1f57db-0b7e-4c3c-a8d8-5bf3005ba1f9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:26.342222 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1f57db-0b7e-4c3c-a8d8-5bf3005ba1f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:26.343851 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1f57db-0b7e-4c3c-a8d8-5bf3005ba1f9 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:26.353026 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 106/219] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 15:59:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:26.354295 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1f57db-0b7e-4c3c-a8d8-5bf3005ba1f9 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:26.361432 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:26.365679 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010c343c-7fe4-41ec-81d2-11512636cd3f None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:26.396729 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:26.408739 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb6cc85-c8d3-4240-838f-846c6a1dcc2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:26.409461 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb6cc85-c8d3-4240-838f-846c6a1dcc2e None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:26.409659 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb6cc85-c8d3-4240-838f-846c6a1dcc2e None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:26.409932 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb6cc85-c8d3-4240-838f-846c6a1dcc2e None admin] PATH_INFO: `/v3/users/5eeb8545f33949879d25acc9fbb7c612` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:26.429047 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb6cc85-c8d3-4240-838f-846c6a1dcc2e None admin] RBAC: Authorizing `identity:get_user(user_id=5eeb8545f33949879d25acc9fbb7c612)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:26.430879 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb6cc85-c8d3-4240-838f-846c6a1dcc2e None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:26.433135 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 114/220] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 19 15:59:26 2022] GET /identity/v3/users/5eeb8545f33949879d25acc9fbb7c612 => generated 297 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:26.434158 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb6cc85-c8d3-4240-838f-846c6a1dcc2e None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:26.490656 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1f57db-0b7e-4c3c-a8d8-5bf3005ba1f9 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:26.531171 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13a401f-b81e-4127-8d4f-ee5dd6b82457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:26.531793 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a401f-b81e-4127-8d4f-ee5dd6b82457 None admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:26.532072 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a401f-b81e-4127-8d4f-ee5dd6b82457 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:26.532343 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a401f-b81e-4127-8d4f-ee5dd6b82457 None admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:26.533003 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13a401f-b81e-4127-8d4f-ee5dd6b82457 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:26.534478 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13a401f-b81e-4127-8d4f-ee5dd6b82457 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:26.550498 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb6cc85-c8d3-4240-838f-846c6a1dcc2e None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:26.568113 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 107/221] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Sep 19 15:59:26 2022] POST /identity/v3/users => generated 297 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 15:59:26.568113 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13a401f-b81e-4127-8d4f-ee5dd6b82457 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:26.602152 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ae03a4-b9f7-4be6-9482-01f1e5d37d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:26.602841 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ae03a4-b9f7-4be6-9482-01f1e5d37d54 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:26.603124 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ae03a4-b9f7-4be6-9482-01f1e5d37d54 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:26.603398 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ae03a4-b9f7-4be6-9482-01f1e5d37d54 None admin] PATH_INFO: `/v3/users/edf78c2cab38448f8c09f5c38ced338b` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:26.604616 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ae03a4-b9f7-4be6-9482-01f1e5d37d54 None admin] RBAC: Authorizing `identity:get_user(user_id=edf78c2cab38448f8c09f5c38ced338b)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:26.606124 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ae03a4-b9f7-4be6-9482-01f1e5d37d54 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:26.608810 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 115/222] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 19 15:59:26 2022] GET /identity/v3/users/edf78c2cab38448f8c09f5c38ced338b => generated 283 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:26.609556 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22f745d-47d3-4758-91c6-76eaf4d49f4c None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:26.609916 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22f745d-47d3-4758-91c6-76eaf4d49f4c None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:26.610195 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22f745d-47d3-4758-91c6-76eaf4d49f4c None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:26.611657 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 116/223] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:59:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 15:59:26.616691 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22f745d-47d3-4758-91c6-76eaf4d49f4c None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:26.647835 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:26.662532 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:26.685056 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13a401f-b81e-4127-8d4f-ee5dd6b82457 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:26.721902 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce96ed1-260a-4315-9d66-7f07f766f23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:26.722566 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce96ed1-260a-4315-9d66-7f07f766f23b None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:26.722848 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce96ed1-260a-4315-9d66-7f07f766f23b None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:26.723121 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce96ed1-260a-4315-9d66-7f07f766f23b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:26.738247 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22f745d-47d3-4758-91c6-76eaf4d49f4c None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:26.790299 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9a8ee2-b644-4b82-bfbf-3d49c9a8117a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:26.791298 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9a8ee2-b644-4b82-bfbf-3d49c9a8117a None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:26.791583 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9a8ee2-b644-4b82-bfbf-3d49c9a8117a None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:26.791857 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9a8ee2-b644-4b82-bfbf-3d49c9a8117a None admin] PATH_INFO: `/v3/projects/93b0db50dd5e4d3195beded37dd4105f` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:26.793014 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9a8ee2-b644-4b82-bfbf-3d49c9a8117a None admin] RBAC: Authorizing `identity:get_project(project_id=93b0db50dd5e4d3195beded37dd4105f)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:26.794551 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9a8ee2-b644-4b82-bfbf-3d49c9a8117a None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:26.796473 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 117/224] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 19 15:59:26 2022] GET /identity/v3/projects/93b0db50dd5e4d3195beded37dd4105f => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:26.797460 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a1ce05-6f4f-413c-85b2-549adcc8249e None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:26.797777 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a1ce05-6f4f-413c-85b2-549adcc8249e None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:26.798008 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a1ce05-6f4f-413c-85b2-549adcc8249e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:26.809137 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce96ed1-260a-4315-9d66-7f07f766f23b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:26.810640 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce96ed1-260a-4315-9d66-7f07f766f23b None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:26.817496 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-45a1ce05-6f4f-413c-85b2-549adcc8249e None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:59:26.818741 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-45a1ce05-6f4f-413c-85b2-549adcc8249e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:59:26.909425 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:26.909615 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 118/225] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:59:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 15:59:26.909693 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a1ce05-6f4f-413c-85b2-549adcc8249e None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:26.931114 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 108/226] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 19 15:59:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 15:59:26.936854 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce96ed1-260a-4315-9d66-7f07f766f23b None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:26.955297 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:27.026524 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a1ce05-6f4f-413c-85b2-549adcc8249e None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:27.058923 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce96ed1-260a-4315-9d66-7f07f766f23b None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:27.065564 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233918c8-77e1-4e2f-b857-afabddfc68f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:27.066238 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233918c8-77e1-4e2f-b857-afabddfc68f5 None admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:27.066572 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233918c8-77e1-4e2f-b857-afabddfc68f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:27.066853 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233918c8-77e1-4e2f-b857-afabddfc68f5 None admin] PATH_INFO: `/v3/projects/93b0db50dd5e4d3195beded37dd4105f/users/edf78c2cab38448f8c09f5c38ced338b/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:27.069085 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233918c8-77e1-4e2f-b857-afabddfc68f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=93b0db50dd5e4d3195beded37dd4105f, user_id=edf78c2cab38448f8c09f5c38ced338b, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:27.070358 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233918c8-77e1-4e2f-b857-afabddfc68f5 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:27.100176 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0059b0c-cf3a-4506-9786-a3ab0fa2e620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:27.101210 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0059b0c-cf3a-4506-9786-a3ab0fa2e620 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:27.101317 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 119/227] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Sep 19 15:59:26 2022] PUT /identity/v3/projects/93b0db50dd5e4d3195beded37dd4105f/users/edf78c2cab38448f8c09f5c38ced338b/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:59:27.101860 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0059b0c-cf3a-4506-9786-a3ab0fa2e620 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:27.102402 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0059b0c-cf3a-4506-9786-a3ab0fa2e620 None admin] PATH_INFO: `/v3/projects/93b0db50dd5e4d3195beded37dd4105f` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:27.103854 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0059b0c-cf3a-4506-9786-a3ab0fa2e620 None admin] RBAC: Authorizing `identity:get_project(project_id=93b0db50dd5e4d3195beded37dd4105f)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:27.105611 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0059b0c-cf3a-4506-9786-a3ab0fa2e620 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:27.108027 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 109/228] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 19 15:59:26 2022] GET /identity/v3/projects/93b0db50dd5e4d3195beded37dd4105f => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:27.115643 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233918c8-77e1-4e2f-b857-afabddfc68f5 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:27.261447 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:27.295268 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233918c8-77e1-4e2f-b857-afabddfc68f5 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:27.335057 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0059b0c-cf3a-4506-9786-a3ab0fa2e620 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:27.341335 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d925114b-a0cb-4e03-a5a6-790c0df24e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:27.341949 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d925114b-a0cb-4e03-a5a6-790c0df24e1c None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:27.342227 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d925114b-a0cb-4e03-a5a6-790c0df24e1c None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:27.342622 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d925114b-a0cb-4e03-a5a6-790c0df24e1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:27.343932 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d925114b-a0cb-4e03-a5a6-790c0df24e1c None admin] RBAC: Adding query filter params (role.id=31b0628699ea4f98824f20e7568748cc, scope.project.id=93b0db50dd5e4d3195beded37dd4105f, user.id=5eeb8545f33949879d25acc9fbb7c612) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:27.344238 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d925114b-a0cb-4e03-a5a6-790c0df24e1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:27.345818 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d925114b-a0cb-4e03-a5a6-790c0df24e1c None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:27.351991 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 120/229] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 19 15:59:27 2022] GET /identity/v3/role_assignments?user.id=5eeb8545f33949879d25acc9fbb7c612&scope.project.id=93b0db50dd5e4d3195beded37dd4105f&role.id=31b0628699ea4f98824f20e7568748cc => generated 259 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:27.402269 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:27.478612 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0059b0c-cf3a-4506-9786-a3ab0fa2e620 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:27.532649 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0e4609-e75e-4ed7-926b-37b05dba0dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:27.533255 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e4609-e75e-4ed7-926b-37b05dba0dad None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:27.533501 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e4609-e75e-4ed7-926b-37b05dba0dad None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:27.533751 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e4609-e75e-4ed7-926b-37b05dba0dad None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:27.550722 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0e4609-e75e-4ed7-926b-37b05dba0dad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 15:59:27.556112 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-6e0e4609-e75e-4ed7-926b-37b05dba0dad None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 15:59:27.556112 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 15:59:27.556112 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 15:59:27.556112 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 15:59:27.556112 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 15:59:27.556112 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 15:59:27.556112 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 15:59:27.556112 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 15:59:27.556112 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 15:59:27.556112 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 15:59:27.556112 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 15:59:27.556112 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 15:59:27.556112 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 15:59:27.556112 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 15:59:27.556112 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 15:59:27.556112 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 15:59:27.557025 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 15:59:27.557025 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 15:59:27.557025 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 15:59:27.557025 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 15:59:27.557025 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 15:59:27.557025 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 15:59:27.557025 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 15:59:27.557025 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 15:59:27.557025 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 15:59:27.557025 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 15:59:27.557025 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 15:59:27.557025 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 15:59:27.557025 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 15:59:27.557025 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 15:59:27.557025 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 15:59:27.557025 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 15:59:27.557792 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 15:59:27.557792 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 15:59:27.557792 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 15:59:27.557792 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 15:59:27.557792 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 15:59:27.557792 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 15:59:27.557792 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 15:59:27.557792 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 15:59:27.557792 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 15:59:27.557792 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 15:59:27.557792 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 15:59:27.557792 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 15:59:27.557792 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 110/230] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Sep 19 15:59:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 15:59:27.564262 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d925114b-a0cb-4e03-a5a6-790c0df24e1c None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:27.712945 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d925114b-a0cb-4e03-a5a6-790c0df24e1c None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:27.772519 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260cb5be-0ab2-4478-9c70-8cf6412a08a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:27.773335 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260cb5be-0ab2-4478-9c70-8cf6412a08a8 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:27.773439 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260cb5be-0ab2-4478-9c70-8cf6412a08a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:27.773671 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260cb5be-0ab2-4478-9c70-8cf6412a08a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:27.774343 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260cb5be-0ab2-4478-9c70-8cf6412a08a8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:27.774679 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260cb5be-0ab2-4478-9c70-8cf6412a08a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:27.777551 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260cb5be-0ab2-4478-9c70-8cf6412a08a8 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:27.785788 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 121/231] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 19 15:59:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:27.792500 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0e4609-e75e-4ed7-926b-37b05dba0dad None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:27.953819 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0e4609-e75e-4ed7-926b-37b05dba0dad None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:27.964466 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:28.005627 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84b62fc-9f03-4814-aa5c-3b2f137e6377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:28.006265 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84b62fc-9f03-4814-aa5c-3b2f137e6377 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:28.006571 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84b62fc-9f03-4814-aa5c-3b2f137e6377 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:28.006839 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84b62fc-9f03-4814-aa5c-3b2f137e6377 None admin] PATH_INFO: `/v3/users/edf78c2cab38448f8c09f5c38ced338b` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:28.007999 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84b62fc-9f03-4814-aa5c-3b2f137e6377 None admin] RBAC: Authorizing `identity:get_user(user_id=edf78c2cab38448f8c09f5c38ced338b)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:28.009565 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84b62fc-9f03-4814-aa5c-3b2f137e6377 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:28.011618 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 111/232] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 19 15:59:27 2022] GET /identity/v3/users/edf78c2cab38448f8c09f5c38ced338b => generated 283 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:28.017292 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260cb5be-0ab2-4478-9c70-8cf6412a08a8 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:28.046116 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:28.181215 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260cb5be-0ab2-4478-9c70-8cf6412a08a8 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:28.232457 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d19a40-c318-44bd-b510-1847ff110808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:28.233100 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d19a40-c318-44bd-b510-1847ff110808 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:28.233911 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d19a40-c318-44bd-b510-1847ff110808 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:28.234184 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d19a40-c318-44bd-b510-1847ff110808 None admin] PATH_INFO: `/v3/projects/93b0db50dd5e4d3195beded37dd4105f` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:28.235370 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d19a40-c318-44bd-b510-1847ff110808 None admin] RBAC: Authorizing `identity:get_project(project_id=93b0db50dd5e4d3195beded37dd4105f)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:28.236828 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d19a40-c318-44bd-b510-1847ff110808 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:28.238488 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 122/233] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 19 15:59:28 2022] GET /identity/v3/projects/93b0db50dd5e4d3195beded37dd4105f => generated 302 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:28.244891 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84b62fc-9f03-4814-aa5c-3b2f137e6377 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:28.415939 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84b62fc-9f03-4814-aa5c-3b2f137e6377 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:28.465803 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a514b7e3-2707-452e-8897-4168b3e8c732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:28.466498 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a514b7e3-2707-452e-8897-4168b3e8c732 None admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:28.466783 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a514b7e3-2707-452e-8897-4168b3e8c732 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:28.467050 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a514b7e3-2707-452e-8897-4168b3e8c732 None admin] PATH_INFO: `/v3/projects/93b0db50dd5e4d3195beded37dd4105f/users/edf78c2cab38448f8c09f5c38ced338b/roles/a1606b79a2c840238de6860bc69b49b7` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:28.472246 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a514b7e3-2707-452e-8897-4168b3e8c732 None admin] RBAC: Authorizing `identity:create_grant(project_id=93b0db50dd5e4d3195beded37dd4105f, user_id=edf78c2cab38448f8c09f5c38ced338b, role_id=a1606b79a2c840238de6860bc69b49b7)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:28.473702 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a514b7e3-2707-452e-8897-4168b3e8c732 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:28.495114 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 112/234] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Sep 19 15:59:28 2022] PUT /identity/v3/projects/93b0db50dd5e4d3195beded37dd4105f/users/edf78c2cab38448f8c09f5c38ced338b/roles/a1606b79a2c840238de6860bc69b49b7 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:59:29.285691 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee0b2a-56fa-448a-a66c-b36414e9825d None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:29.286246 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee0b2a-56fa-448a-a66c-b36414e9825d None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:29.286246 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee0b2a-56fa-448a-a66c-b36414e9825d None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:29.287978 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 123/235] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:59:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 15:59:29.294278 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa12c64-7a59-4dc5-a2ee-2dc3e9d6f002 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:29.294782 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa12c64-7a59-4dc5-a2ee-2dc3e9d6f002 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:29.294857 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa12c64-7a59-4dc5-a2ee-2dc3e9d6f002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:29.315158 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-faa12c64-7a59-4dc5-a2ee-2dc3e9d6f002 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:59:29.316369 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-faa12c64-7a59-4dc5-a2ee-2dc3e9d6f002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:59:29.360414 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:29.420663 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 113/236] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:59:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 15:59:29.558651 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b56657-a6e6-44c5-bb3d-9cc9191d34ec None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:29.558943 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b56657-a6e6-44c5-bb3d-9cc9191d34ec None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:29.559128 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b56657-a6e6-44c5-bb3d-9cc9191d34ec None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:29.560632 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 124/237] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:59:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 15:59:29.566600 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1337194-f5d0-406e-874e-d16e40fb0163 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:29.566888 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1337194-f5d0-406e-874e-d16e40fb0163 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:29.567165 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1337194-f5d0-406e-874e-d16e40fb0163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:29.585691 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-f1337194-f5d0-406e-874e-d16e40fb0163 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:59:29.586931 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-f1337194-f5d0-406e-874e-d16e40fb0163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:59:29.656481 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 114/238] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:59:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 15:59:29.810503 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b56657-a6e6-44c5-bb3d-9cc9191d34ec None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:29.925797 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:29.941508 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1337194-f5d0-406e-874e-d16e40fb0163 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:29.951502 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b56657-a6e6-44c5-bb3d-9cc9191d34ec None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:29.993897 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61511fd0-e284-44f4-bb4c-776335a95bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:29.994572 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61511fd0-e284-44f4-bb4c-776335a95bc5 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:29.994917 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61511fd0-e284-44f4-bb4c-776335a95bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:29.995504 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61511fd0-e284-44f4-bb4c-776335a95bc5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:30.010498 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61511fd0-e284-44f4-bb4c-776335a95bc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 15:59:30.018000 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-61511fd0-e284-44f4-bb4c-776335a95bc5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 15:59:30.018000 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 15:59:30.018000 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 15:59:30.018000 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 15:59:30.018000 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 15:59:30.018000 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 15:59:30.018000 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 15:59:30.018000 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 15:59:30.018000 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 15:59:30.018000 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 15:59:30.018000 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 15:59:30.018000 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 15:59:30.018000 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 15:59:30.018000 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 15:59:30.018000 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 15:59:30.018000 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 15:59:30.018937 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 15:59:30.018937 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 15:59:30.018937 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 15:59:30.018937 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 15:59:30.018937 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 15:59:30.018937 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 15:59:30.018937 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 15:59:30.018937 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 15:59:30.018937 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 15:59:30.018937 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 15:59:30.018937 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 15:59:30.018937 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 15:59:30.018937 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 15:59:30.018937 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 15:59:30.018937 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 15:59:30.018937 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 15:59:30.019727 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 15:59:30.019727 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 15:59:30.019727 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 15:59:30.019727 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 15:59:30.019727 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 15:59:30.019727 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 15:59:30.019727 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 15:59:30.019727 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 15:59:30.019727 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 15:59:30.019727 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 15:59:30.019727 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 15:59:30.019727 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 15:59:30.020324 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 125/239] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 15:59:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 15:59:30.024629 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b6e3a7-c5f1-4be5-b065-d997d3adf147 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:30.024629 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b6e3a7-c5f1-4be5-b065-d997d3adf147 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:30.024629 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b6e3a7-c5f1-4be5-b065-d997d3adf147 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:30.024629 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 126/240] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:59:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 15:59:30.025305 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b6e3a7-c5f1-4be5-b065-d997d3adf147 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:30.055952 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:30.060155 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1337194-f5d0-406e-874e-d16e40fb0163 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:30.102187 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdea082-a848-4569-95fb-e365228c4974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:30.102972 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdea082-a848-4569-95fb-e365228c4974 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:30.103330 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdea082-a848-4569-95fb-e365228c4974 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:30.103678 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdea082-a848-4569-95fb-e365228c4974 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:30.104936 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdea082-a848-4569-95fb-e365228c4974 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:30.106555 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdea082-a848-4569-95fb-e365228c4974 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:30.113514 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 115/241] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 15:59:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:30.115088 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c35e9e-79a8-44e8-9f2c-763b88c2dd62 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:30.115530 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c35e9e-79a8-44e8-9f2c-763b88c2dd62 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:30.115925 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c35e9e-79a8-44e8-9f2c-763b88c2dd62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:30.134325 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-45c35e9e-79a8-44e8-9f2c-763b88c2dd62 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:59:30.135689 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-45c35e9e-79a8-44e8-9f2c-763b88c2dd62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:59:30.169546 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b6e3a7-c5f1-4be5-b065-d997d3adf147 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:30.214462 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 116/242] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:59:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 15:59:30.215538 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c35e9e-79a8-44e8-9f2c-763b88c2dd62 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:30.224925 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802e6699-4946-40cd-bb1f-733ad74c546b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:30.225554 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802e6699-4946-40cd-bb1f-733ad74c546b None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:30.225831 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802e6699-4946-40cd-bb1f-733ad74c546b None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:30.226100 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802e6699-4946-40cd-bb1f-733ad74c546b None admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:30.226922 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802e6699-4946-40cd-bb1f-733ad74c546b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:30.227219 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802e6699-4946-40cd-bb1f-733ad74c546b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:30.228726 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802e6699-4946-40cd-bb1f-733ad74c546b None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:30.235209 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:30.247110 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 127/243] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Sep 19 15:59:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 15:59:30.257852 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802e6699-4946-40cd-bb1f-733ad74c546b None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:30.350772 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:30.362512 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c35e9e-79a8-44e8-9f2c-763b88c2dd62 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:30.404853 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd29b169-31b0-4a28-b272-7c0fa9128ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:30.405588 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd29b169-31b0-4a28-b272-7c0fa9128ffd None admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:30.405871 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd29b169-31b0-4a28-b272-7c0fa9128ffd None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:30.406261 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd29b169-31b0-4a28-b272-7c0fa9128ffd None admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:30.406949 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd29b169-31b0-4a28-b272-7c0fa9128ffd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:30.408506 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd29b169-31b0-4a28-b272-7c0fa9128ffd None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:30.429171 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802e6699-4946-40cd-bb1f-733ad74c546b None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:30.433916 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 117/244] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Sep 19 15:59:30 2022] POST /identity/v3/users => generated 293 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 15:59:30.435344 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2735846-3a2c-47fc-84bd-11150bf35f4b None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:30.435681 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2735846-3a2c-47fc-84bd-11150bf35f4b None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:30.435954 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2735846-3a2c-47fc-84bd-11150bf35f4b None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:30.443484 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 118/245] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:59:30 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 15:59:30.445495 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fade5dfa-f764-4616-a55d-19f3d67648aa None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:30.445783 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fade5dfa-f764-4616-a55d-19f3d67648aa None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:30.446383 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fade5dfa-f764-4616-a55d-19f3d67648aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:30.475110 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-fade5dfa-f764-4616-a55d-19f3d67648aa None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:59:30.478149 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-fade5dfa-f764-4616-a55d-19f3d67648aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:59:30.497142 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e69842a-c672-41dc-ae0e-052e04cb3280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:30.497916 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69842a-c672-41dc-ae0e-052e04cb3280 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:30.498199 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69842a-c672-41dc-ae0e-052e04cb3280 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:30.498490 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69842a-c672-41dc-ae0e-052e04cb3280 None admin] PATH_INFO: `/v3/users/ef6670387b9242ac857743e91aadc2f9` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:30.533325 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e69842a-c672-41dc-ae0e-052e04cb3280 None admin] RBAC: Authorizing `identity:get_user(user_id=ef6670387b9242ac857743e91aadc2f9)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:30.536621 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e69842a-c672-41dc-ae0e-052e04cb3280 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:30.538405 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 128/246] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 19 15:59:30 2022] GET /identity/v3/users/ef6670387b9242ac857743e91aadc2f9 => generated 297 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 15:59:30.555239 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e69842a-c672-41dc-ae0e-052e04cb3280 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:30.604811 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 119/247] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:59:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 15:59:30.605685 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fade5dfa-f764-4616-a55d-19f3d67648aa None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:30.635423 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:30.687773 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e69842a-c672-41dc-ae0e-052e04cb3280 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:30.734420 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:30.734796 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3065de16-20b6-4259-a3c0-8bab822994a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:30.735751 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3065de16-20b6-4259-a3c0-8bab822994a7 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:30.736243 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3065de16-20b6-4259-a3c0-8bab822994a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:30.736654 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3065de16-20b6-4259-a3c0-8bab822994a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:30.760689 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fade5dfa-f764-4616-a55d-19f3d67648aa None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:30.810540 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8869a4ec-400d-45d9-aa4b-6a457f97daaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:30.811279 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869a4ec-400d-45d9-aa4b-6a457f97daaa None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:30.811566 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869a4ec-400d-45d9-aa4b-6a457f97daaa None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:30.811899 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869a4ec-400d-45d9-aa4b-6a457f97daaa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:30.828191 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8869a4ec-400d-45d9-aa4b-6a457f97daaa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 15:59:30.833385 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-8869a4ec-400d-45d9-aa4b-6a457f97daaa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 15:59:30.833385 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 15:59:30.833385 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 15:59:30.833385 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 15:59:30.833385 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 15:59:30.833385 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 15:59:30.833385 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 15:59:30.833385 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 15:59:30.833385 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 15:59:30.833385 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 15:59:30.833385 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 15:59:30.833385 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 15:59:30.833385 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 15:59:30.833385 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 15:59:30.833385 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 15:59:30.833385 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 15:59:30.834364 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 15:59:30.834364 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 15:59:30.834364 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 15:59:30.834364 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 15:59:30.834364 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 15:59:30.834364 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 15:59:30.834364 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 15:59:30.834364 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 15:59:30.834364 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 15:59:30.834364 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 15:59:30.834364 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 15:59:30.834364 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 15:59:30.834364 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 15:59:30.834364 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 15:59:30.834364 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 15:59:30.834364 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 15:59:30.835188 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 15:59:30.835188 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 15:59:30.835188 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 15:59:30.835188 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 15:59:30.835188 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 15:59:30.835188 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 15:59:30.835188 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 15:59:30.835188 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 15:59:30.835188 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 15:59:30.835188 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 15:59:30.835188 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 15:59:30.835188 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 15:59:30.835188 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 120/248] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 19 15:59:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 15:59:30.842349 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8869a4ec-400d-45d9-aa4b-6a457f97daaa None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:30.851627 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3065de16-20b6-4259-a3c0-8bab822994a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:30.853507 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3065de16-20b6-4259-a3c0-8bab822994a7 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:30.942953 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:30.968374 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8869a4ec-400d-45d9-aa4b-6a457f97daaa None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:31.002743 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 129/249] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 19 15:59:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 19 15:59:31.004426 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3065de16-20b6-4259-a3c0-8bab822994a7 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:31.012785 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d98a9b-c7c3-409d-a271-6a13db20cb9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:31.013410 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d98a9b-c7c3-409d-a271-6a13db20cb9e None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:31.013692 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d98a9b-c7c3-409d-a271-6a13db20cb9e None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:31.013996 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d98a9b-c7c3-409d-a271-6a13db20cb9e None admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:31.014893 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d98a9b-c7c3-409d-a271-6a13db20cb9e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:31.015065 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d98a9b-c7c3-409d-a271-6a13db20cb9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:31.016459 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d98a9b-c7c3-409d-a271-6a13db20cb9e None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:31.026897 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 121/250] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 15:59:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:31.027570 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d98a9b-c7c3-409d-a271-6a13db20cb9e None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:31.032053 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:31.159082 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3065de16-20b6-4259-a3c0-8bab822994a7 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:31.188788 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d98a9b-c7c3-409d-a271-6a13db20cb9e None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:31.209634 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574b3bf0-475a-40d5-bac4-8ec07190ab87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:31.210034 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574b3bf0-475a-40d5-bac4-8ec07190ab87 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:31.210346 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574b3bf0-475a-40d5-bac4-8ec07190ab87 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:31.210691 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574b3bf0-475a-40d5-bac4-8ec07190ab87 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:31.223811 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574b3bf0-475a-40d5-bac4-8ec07190ab87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 15:59:31.229812 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-574b3bf0-475a-40d5-bac4-8ec07190ab87 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 15:59:31.229812 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 15:59:31.229812 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 15:59:31.229812 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 15:59:31.229812 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 15:59:31.229812 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 15:59:31.229812 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 15:59:31.229812 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 15:59:31.229812 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 15:59:31.229812 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 15:59:31.229812 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 15:59:31.229812 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 15:59:31.229812 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 15:59:31.229812 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 15:59:31.229812 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 15:59:31.229812 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 15:59:31.230804 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 15:59:31.230804 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 15:59:31.230804 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 15:59:31.230804 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 15:59:31.230804 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 15:59:31.230804 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 15:59:31.230804 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 15:59:31.230804 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 15:59:31.230804 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 15:59:31.230804 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 15:59:31.230804 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 15:59:31.230804 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 15:59:31.230804 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 15:59:31.230804 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 15:59:31.230804 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 15:59:31.230804 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 15:59:31.232032 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 15:59:31.232032 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 15:59:31.232032 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 15:59:31.232032 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 15:59:31.232032 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 15:59:31.232032 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 15:59:31.232032 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 15:59:31.232032 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 15:59:31.232032 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 15:59:31.232032 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 15:59:31.232032 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 15:59:31.232032 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 15:59:31.232032 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 130/251] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 15:59:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 15:59:31.232672 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574b3bf0-475a-40d5-bac4-8ec07190ab87 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:31.244742 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40faefd5-878d-4dc8-b62e-c71cee8968d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:31.245788 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40faefd5-878d-4dc8-b62e-c71cee8968d0 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:31.246273 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40faefd5-878d-4dc8-b62e-c71cee8968d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:31.250918 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40faefd5-878d-4dc8-b62e-c71cee8968d0 None admin] PATH_INFO: `/v3/projects/93b0db50dd5e4d3195beded37dd4105f` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:31.253379 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40faefd5-878d-4dc8-b62e-c71cee8968d0 None admin] RBAC: Authorizing `identity:get_project(project_id=93b0db50dd5e4d3195beded37dd4105f)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:31.254103 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40faefd5-878d-4dc8-b62e-c71cee8968d0 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:31.258974 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 122/252] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 19 15:59:31 2022] GET /identity/v3/projects/93b0db50dd5e4d3195beded37dd4105f => generated 302 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:31.258974 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40faefd5-878d-4dc8-b62e-c71cee8968d0 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:31.276903 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:31.366544 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:31.400807 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574b3bf0-475a-40d5-bac4-8ec07190ab87 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:31.403265 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40faefd5-878d-4dc8-b62e-c71cee8968d0 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:31.445492 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee6e629-b692-45bd-bab6-8083ea970a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:31.446512 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee6e629-b692-45bd-bab6-8083ea970a15 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:31.446998 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee6e629-b692-45bd-bab6-8083ea970a15 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:31.447470 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee6e629-b692-45bd-bab6-8083ea970a15 None admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:31.448359 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee6e629-b692-45bd-bab6-8083ea970a15 None admin] RBAC: Adding query filter params (name=member) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:31.448866 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee6e629-b692-45bd-bab6-8083ea970a15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:31.449170 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d1542f-0509-44a7-9950-c8d9665f6a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:31.449811 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1542f-0509-44a7-9950-c8d9665f6a14 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:31.450095 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1542f-0509-44a7-9950-c8d9665f6a14 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:31.450400 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1542f-0509-44a7-9950-c8d9665f6a14 None admin] PATH_INFO: `/v3/users/edf78c2cab38448f8c09f5c38ced338b` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:31.451578 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d1542f-0509-44a7-9950-c8d9665f6a14 None admin] RBAC: Authorizing `identity:get_user(user_id=edf78c2cab38448f8c09f5c38ced338b)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:31.452094 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee6e629-b692-45bd-bab6-8083ea970a15 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:31.453130 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d1542f-0509-44a7-9950-c8d9665f6a14 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:31.455653 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 131/253] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 19 15:59:31 2022] GET /identity/v3/users/edf78c2cab38448f8c09f5c38ced338b => generated 283 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:31.456466 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d1542f-0509-44a7-9950-c8d9665f6a14 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:31.463821 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 123/254] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Sep 19 15:59:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:31.465460 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0692591a-bf2f-4c61-a4db-f4ab9590622a None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:31.465950 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0692591a-bf2f-4c61-a4db-f4ab9590622a None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:31.466461 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0692591a-bf2f-4c61-a4db-f4ab9590622a None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:31.468848 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 124/255] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:59:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 15:59:31.469801 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0692591a-bf2f-4c61-a4db-f4ab9590622a None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:31.593169 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:31.597233 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0692591a-bf2f-4c61-a4db-f4ab9590622a None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:31.610240 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d1542f-0509-44a7-9950-c8d9665f6a14 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:31.638818 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4332cdbd-48ac-4d9c-9e09-ea854b613db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:31.640206 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4332cdbd-48ac-4d9c-9e09-ea854b613db2 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:31.640554 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4332cdbd-48ac-4d9c-9e09-ea854b613db2 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:31.640894 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4332cdbd-48ac-4d9c-9e09-ea854b613db2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:31.644380 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:31.650255 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da7cae8-57f8-482e-8d72-b543207d4bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:31.651204 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da7cae8-57f8-482e-8d72-b543207d4bbb None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:31.651668 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da7cae8-57f8-482e-8d72-b543207d4bbb None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:31.652123 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da7cae8-57f8-482e-8d72-b543207d4bbb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:31.653091 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da7cae8-57f8-482e-8d72-b543207d4bbb None admin] RBAC: Adding query filter params (role.id=625d911d4d934c1e80f0a57620077d49, scope.project.id=93b0db50dd5e4d3195beded37dd4105f, user.id=ef6670387b9242ac857743e91aadc2f9) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:31.653584 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da7cae8-57f8-482e-8d72-b543207d4bbb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:31.655909 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da7cae8-57f8-482e-8d72-b543207d4bbb None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:31.662153 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 132/256] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 19 15:59:31 2022] GET /identity/v3/role_assignments?user.id=ef6670387b9242ac857743e91aadc2f9&scope.project.id=93b0db50dd5e4d3195beded37dd4105f&role.id=625d911d4d934c1e80f0a57620077d49 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:31.663608 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a90a7-835a-43a2-bb05-bf9e129b783c None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:31.664041 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a90a7-835a-43a2-bb05-bf9e129b783c None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:31.664598 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a90a7-835a-43a2-bb05-bf9e129b783c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:31.685363 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-249a90a7-835a-43a2-bb05-bf9e129b783c None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:59:31.686872 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-249a90a7-835a-43a2-bb05-bf9e129b783c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:59:31.728797 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4332cdbd-48ac-4d9c-9e09-ea854b613db2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:31.730586 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4332cdbd-48ac-4d9c-9e09-ea854b613db2 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:31.826648 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 133/257] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:59:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 15:59:31.829039 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249a90a7-835a-43a2-bb05-bf9e129b783c None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:31.953086 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 125/258] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 19 15:59:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 15:59:31.959070 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4332cdbd-48ac-4d9c-9e09-ea854b613db2 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:32.004648 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:32.021613 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:32.027175 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249a90a7-835a-43a2-bb05-bf9e129b783c None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:32.095069 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4e735f-dfb8-4642-82c8-a6f3a00a3334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:32.095727 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4e735f-dfb8-4642-82c8-a6f3a00a3334 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:32.096005 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4e735f-dfb8-4642-82c8-a6f3a00a3334 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:32.096277 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4e735f-dfb8-4642-82c8-a6f3a00a3334 None admin] PATH_INFO: `/v3/users/5eeb8545f33949879d25acc9fbb7c612` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:32.099171 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4e735f-dfb8-4642-82c8-a6f3a00a3334 None admin] RBAC: Authorizing `identity:get_user(user_id=5eeb8545f33949879d25acc9fbb7c612)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:32.100027 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4332cdbd-48ac-4d9c-9e09-ea854b613db2 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:32.101020 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4e735f-dfb8-4642-82c8-a6f3a00a3334 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:32.103150 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 134/259] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 19 15:59:31 2022] GET /identity/v3/users/5eeb8545f33949879d25acc9fbb7c612 => generated 297 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:32.108970 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4e735f-dfb8-4642-82c8-a6f3a00a3334 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:32.157009 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f66700f-fa6c-4d83-8b37-8d843f2787db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:32.157644 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f66700f-fa6c-4d83-8b37-8d843f2787db None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:32.157921 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f66700f-fa6c-4d83-8b37-8d843f2787db None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:32.158193 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f66700f-fa6c-4d83-8b37-8d843f2787db None admin] PATH_INFO: `/v3/projects/93b0db50dd5e4d3195beded37dd4105f` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:32.163073 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f66700f-fa6c-4d83-8b37-8d843f2787db None admin] RBAC: Authorizing `identity:get_project(project_id=93b0db50dd5e4d3195beded37dd4105f)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:32.164508 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f66700f-fa6c-4d83-8b37-8d843f2787db None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:32.166825 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 126/260] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 19 15:59:31 2022] GET /identity/v3/projects/93b0db50dd5e4d3195beded37dd4105f => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:32.172803 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f66700f-fa6c-4d83-8b37-8d843f2787db None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:32.262830 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4e735f-dfb8-4642-82c8-a6f3a00a3334 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:32.312453 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40715df-5f56-4a9d-9166-87c4840d9f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:32.313482 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40715df-5f56-4a9d-9166-87c4840d9f17 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:32.313906 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40715df-5f56-4a9d-9166-87c4840d9f17 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:32.314478 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40715df-5f56-4a9d-9166-87c4840d9f17 None admin] PATH_INFO: `/v3/projects/93b0db50dd5e4d3195beded37dd4105f` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:32.315979 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40715df-5f56-4a9d-9166-87c4840d9f17 None admin] RBAC: Authorizing `identity:get_project(project_id=93b0db50dd5e4d3195beded37dd4105f)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:32.317601 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40715df-5f56-4a9d-9166-87c4840d9f17 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:32.320339 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 135/261] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 19 15:59:32 2022] GET /identity/v3/projects/93b0db50dd5e4d3195beded37dd4105f => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:32.321427 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40715df-5f56-4a9d-9166-87c4840d9f17 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:32.360006 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:32.415038 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f66700f-fa6c-4d83-8b37-8d843f2787db None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:32.418690 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:32.486855 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b373ad92-2976-4c88-94c8-1036923a5f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:32.487895 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b373ad92-2976-4c88-94c8-1036923a5f20 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:32.488184 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b373ad92-2976-4c88-94c8-1036923a5f20 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:32.488477 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b373ad92-2976-4c88-94c8-1036923a5f20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:32.489361 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b373ad92-2976-4c88-94c8-1036923a5f20 None admin] RBAC: Adding query filter params (role.id=63d2d197bfd6497583e94bfb065ce29b, scope.project.id=93b0db50dd5e4d3195beded37dd4105f, user.id=edf78c2cab38448f8c09f5c38ced338b) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:32.489677 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b373ad92-2976-4c88-94c8-1036923a5f20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:32.491348 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b373ad92-2976-4c88-94c8-1036923a5f20 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:32.498416 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 127/262] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 19 15:59:32 2022] GET /identity/v3/role_assignments?user.id=edf78c2cab38448f8c09f5c38ced338b&scope.project.id=93b0db50dd5e4d3195beded37dd4105f&role.id=63d2d197bfd6497583e94bfb065ce29b => generated 608 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 15:59:32.499852 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b373ad92-2976-4c88-94c8-1036923a5f20 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:32.518837 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40715df-5f56-4a9d-9166-87c4840d9f17 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:32.556742 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c246109a-0f81-4a0c-b326-2cd46dfa5832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:32.557441 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c246109a-0f81-4a0c-b326-2cd46dfa5832 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:32.557727 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c246109a-0f81-4a0c-b326-2cd46dfa5832 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:32.558003 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c246109a-0f81-4a0c-b326-2cd46dfa5832 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:32.570637 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c246109a-0f81-4a0c-b326-2cd46dfa5832 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 15:59:32.576229 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-c246109a-0f81-4a0c-b326-2cd46dfa5832 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 15:59:32.576229 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 15:59:32.576229 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 15:59:32.576229 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 15:59:32.576229 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 15:59:32.576229 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 15:59:32.576229 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 15:59:32.576229 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 15:59:32.576229 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 15:59:32.576229 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 15:59:32.576229 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 15:59:32.576229 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 15:59:32.576229 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 15:59:32.576229 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 15:59:32.576229 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 15:59:32.576229 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 15:59:32.577676 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 15:59:32.577676 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 15:59:32.577676 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 15:59:32.577676 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 15:59:32.577676 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 15:59:32.577676 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 15:59:32.577676 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 15:59:32.577676 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 15:59:32.577676 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 15:59:32.577676 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 15:59:32.577676 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 15:59:32.577676 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 15:59:32.577676 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 15:59:32.577676 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 15:59:32.577676 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 15:59:32.577676 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 15:59:32.578561 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 15:59:32.578561 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 15:59:32.578561 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 15:59:32.578561 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 15:59:32.578561 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 15:59:32.578561 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 15:59:32.578561 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 15:59:32.578561 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 15:59:32.578561 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 15:59:32.578561 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 15:59:32.578561 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 15:59:32.578561 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 15:59:32.578561 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 136/263] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Sep 19 15:59:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 15:59:32.583995 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c246109a-0f81-4a0c-b326-2cd46dfa5832 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:32.637299 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b373ad92-2976-4c88-94c8-1036923a5f20 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:32.679863 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e39127-2d09-478b-a38d-f7c655905ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:32.680550 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e39127-2d09-478b-a38d-f7c655905ddd None admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:32.680833 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e39127-2d09-478b-a38d-f7c655905ddd None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:32.681226 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e39127-2d09-478b-a38d-f7c655905ddd None admin] PATH_INFO: `/v3/projects/93b0db50dd5e4d3195beded37dd4105f/users/5eeb8545f33949879d25acc9fbb7c612/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:32.688694 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e39127-2d09-478b-a38d-f7c655905ddd None admin] RBAC: Authorizing `identity:create_grant(project_id=93b0db50dd5e4d3195beded37dd4105f, user_id=5eeb8545f33949879d25acc9fbb7c612, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:32.690177 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e39127-2d09-478b-a38d-f7c655905ddd None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:32.717556 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 128/264] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Sep 19 15:59:32 2022] PUT /identity/v3/projects/93b0db50dd5e4d3195beded37dd4105f/users/5eeb8545f33949879d25acc9fbb7c612/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:59:32.734804 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:32.758795 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c246109a-0f81-4a0c-b326-2cd46dfa5832 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:32.799867 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beed3c6f-fbba-49c8-bf96-6e23d9af6342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:32.800976 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beed3c6f-fbba-49c8-bf96-6e23d9af6342 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:32.801258 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beed3c6f-fbba-49c8-bf96-6e23d9af6342 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:32.801621 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beed3c6f-fbba-49c8-bf96-6e23d9af6342 None admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:32.802332 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beed3c6f-fbba-49c8-bf96-6e23d9af6342 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:32.802676 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beed3c6f-fbba-49c8-bf96-6e23d9af6342 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:32.804376 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beed3c6f-fbba-49c8-bf96-6e23d9af6342 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:32.814902 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 137/265] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 19 15:59:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 15:59:32.820641 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e39127-2d09-478b-a38d-f7c655905ddd None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:32.854514 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:32.949701 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e39127-2d09-478b-a38d-f7c655905ddd None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:32.988582 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6686f148-8b50-4860-b574-6110d7dbb13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:32.989246 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6686f148-8b50-4860-b574-6110d7dbb13d None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:32.989530 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6686f148-8b50-4860-b574-6110d7dbb13d None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:32.989807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6686f148-8b50-4860-b574-6110d7dbb13d None admin] PATH_INFO: `/v3/users/edf78c2cab38448f8c09f5c38ced338b` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:32.991118 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6686f148-8b50-4860-b574-6110d7dbb13d None admin] RBAC: Authorizing `identity:get_user(user_id=edf78c2cab38448f8c09f5c38ced338b)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:32.992640 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6686f148-8b50-4860-b574-6110d7dbb13d None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:32.994921 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 129/266] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 19 15:59:32 2022] GET /identity/v3/users/edf78c2cab38448f8c09f5c38ced338b => generated 283 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:33.001346 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beed3c6f-fbba-49c8-bf96-6e23d9af6342 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:33.129516 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beed3c6f-fbba-49c8-bf96-6e23d9af6342 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:33.171844 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4effc3c4-a979-482e-aef9-e860964cb8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:33.172466 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4effc3c4-a979-482e-aef9-e860964cb8e3 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:33.172748 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4effc3c4-a979-482e-aef9-e860964cb8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:33.173022 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4effc3c4-a979-482e-aef9-e860964cb8e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:33.183720 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:33.252141 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4effc3c4-a979-482e-aef9-e860964cb8e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:33.253644 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4effc3c4-a979-482e-aef9-e860964cb8e3 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:33.395726 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 138/267] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 19 15:59:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 15:59:33.401108 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6686f148-8b50-4860-b574-6110d7dbb13d None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:33.512355 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:33.520079 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6686f148-8b50-4860-b574-6110d7dbb13d None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:33.557705 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5782ae-6cc8-40ab-b890-9f37058f5443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:33.558352 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5782ae-6cc8-40ab-b890-9f37058f5443 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:33.558663 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5782ae-6cc8-40ab-b890-9f37058f5443 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:33.558940 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5782ae-6cc8-40ab-b890-9f37058f5443 None admin] PATH_INFO: `/v3/projects/93b0db50dd5e4d3195beded37dd4105f` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:33.560117 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5782ae-6cc8-40ab-b890-9f37058f5443 None admin] RBAC: Authorizing `identity:get_project(project_id=93b0db50dd5e4d3195beded37dd4105f)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:33.562207 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5782ae-6cc8-40ab-b890-9f37058f5443 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:33.564055 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 130/268] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 19 15:59:33 2022] GET /identity/v3/projects/93b0db50dd5e4d3195beded37dd4105f => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:33.569959 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4effc3c4-a979-482e-aef9-e860964cb8e3 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:33.615078 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:33.675025 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e64315c-8d1d-47b9-b891-38e129e2cbed None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:33.675195 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e64315c-8d1d-47b9-b891-38e129e2cbed None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:33.675477 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e64315c-8d1d-47b9-b891-38e129e2cbed None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:33.677447 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 131/269] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:59:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 15:59:33.685981 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b902cb-2987-4e74-8a25-0af5f383f62f None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:33.686947 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b902cb-2987-4e74-8a25-0af5f383f62f None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:33.687529 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b902cb-2987-4e74-8a25-0af5f383f62f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:33.692561 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4effc3c4-a979-482e-aef9-e860964cb8e3 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:33.717305 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-f8b902cb-2987-4e74-8a25-0af5f383f62f None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:59:33.718636 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-f8b902cb-2987-4e74-8a25-0af5f383f62f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:59:33.747945 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d37f02-3adf-40fb-a269-8d81448b4767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:33.749081 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d37f02-3adf-40fb-a269-8d81448b4767 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:33.749375 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d37f02-3adf-40fb-a269-8d81448b4767 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:33.749641 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d37f02-3adf-40fb-a269-8d81448b4767 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:33.750406 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d37f02-3adf-40fb-a269-8d81448b4767 None admin] RBAC: Adding query filter params (role.id=a1606b79a2c840238de6860bc69b49b7, scope.project.id=93b0db50dd5e4d3195beded37dd4105f, user.id=edf78c2cab38448f8c09f5c38ced338b) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:33.750716 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d37f02-3adf-40fb-a269-8d81448b4767 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:33.752191 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d37f02-3adf-40fb-a269-8d81448b4767 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:33.765859 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 139/270] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 19 15:59:33 2022] GET /identity/v3/role_assignments?user.id=edf78c2cab38448f8c09f5c38ced338b&scope.project.id=93b0db50dd5e4d3195beded37dd4105f&role.id=a1606b79a2c840238de6860bc69b49b7 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:33.880256 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 132/271] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:59:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 15:59:34.113476 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2ed4f1-5709-4b31-96e0-735f8fa31b71 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:34.113965 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2ed4f1-5709-4b31-96e0-735f8fa31b71 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:34.114554 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2ed4f1-5709-4b31-96e0-735f8fa31b71 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:34.117668 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 140/272] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:59:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 15:59:34.123271 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556786d2-77b5-41a1-b825-de86619f8934 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:34.123808 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556786d2-77b5-41a1-b825-de86619f8934 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:34.124222 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556786d2-77b5-41a1-b825-de86619f8934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:34.154428 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-556786d2-77b5-41a1-b825-de86619f8934 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:59:34.157012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-556786d2-77b5-41a1-b825-de86619f8934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:59:34.284579 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 133/273] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:59:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 15:59:34.335206 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2ed4f1-5709-4b31-96e0-735f8fa31b71 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:34.455432 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2ed4f1-5709-4b31-96e0-735f8fa31b71 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:34.457124 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:34.491930 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730a72e9-25bc-45d7-87ad-add6ec12905c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:34.492629 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730a72e9-25bc-45d7-87ad-add6ec12905c None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:34.492950 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730a72e9-25bc-45d7-87ad-add6ec12905c None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:34.493230 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730a72e9-25bc-45d7-87ad-add6ec12905c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:34.504024 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730a72e9-25bc-45d7-87ad-add6ec12905c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 15:59:34.509149 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-730a72e9-25bc-45d7-87ad-add6ec12905c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 15:59:34.509149 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 15:59:34.509149 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 15:59:34.509149 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 15:59:34.509149 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 15:59:34.509149 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 15:59:34.509149 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 15:59:34.509149 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 15:59:34.509149 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 15:59:34.509149 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 15:59:34.509149 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 15:59:34.509149 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 15:59:34.509149 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 15:59:34.509149 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 15:59:34.509149 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 15:59:34.509149 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 15:59:34.509966 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 15:59:34.509966 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 15:59:34.509966 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 15:59:34.509966 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 15:59:34.509966 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 15:59:34.509966 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 15:59:34.509966 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 15:59:34.509966 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 15:59:34.509966 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 15:59:34.509966 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 15:59:34.509966 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 15:59:34.509966 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 15:59:34.509966 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 15:59:34.509966 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 15:59:34.509966 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 15:59:34.509966 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 15:59:34.511160 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 15:59:34.511160 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 15:59:34.511160 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 15:59:34.511160 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 15:59:34.511160 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 15:59:34.511160 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 15:59:34.511160 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 15:59:34.511160 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 15:59:34.511160 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 15:59:34.511160 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 15:59:34.511160 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 15:59:34.511160 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 15:59:34.511160 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 141/274] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 15:59:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 15:59:34.516241 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556786d2-77b5-41a1-b825-de86619f8934 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:34.521104 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ea232-56ab-4997-bdd7-90ae1a243da2 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:34.521389 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ea232-56ab-4997-bdd7-90ae1a243da2 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:34.521663 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ea232-56ab-4997-bdd7-90ae1a243da2 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:34.523309 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 142/275] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:59:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 15:59:34.530322 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24236e1d-cc2c-4700-a7d0-7355c85aefce None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:34.530641 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24236e1d-cc2c-4700-a7d0-7355c85aefce None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:34.530916 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24236e1d-cc2c-4700-a7d0-7355c85aefce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:34.549000 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:34.551637 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-24236e1d-cc2c-4700-a7d0-7355c85aefce None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:59:34.552878 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-24236e1d-cc2c-4700-a7d0-7355c85aefce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:59:34.631122 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556786d2-77b5-41a1-b825-de86619f8934 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:34.658949 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 143/276] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:59:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 15:59:34.662906 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24236e1d-cc2c-4700-a7d0-7355c85aefce None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:34.685115 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15bd621-07f7-4846-8ede-1516b8eeeab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:34.685758 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15bd621-07f7-4846-8ede-1516b8eeeab1 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:34.686426 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15bd621-07f7-4846-8ede-1516b8eeeab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:34.686717 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15bd621-07f7-4846-8ede-1516b8eeeab1 None admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:34.687983 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15bd621-07f7-4846-8ede-1516b8eeeab1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:34.688413 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15bd621-07f7-4846-8ede-1516b8eeeab1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:34.690611 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15bd621-07f7-4846-8ede-1516b8eeeab1 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:34.703051 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 134/277] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Sep 19 15:59:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:34.708428 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15bd621-07f7-4846-8ede-1516b8eeeab1 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:34.765779 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:34.784834 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24236e1d-cc2c-4700-a7d0-7355c85aefce None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:34.833043 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4c804c-da64-4528-bf8a-d20753d1af86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:34.833497 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15bd621-07f7-4846-8ede-1516b8eeeab1 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:34.834282 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4c804c-da64-4528-bf8a-d20753d1af86 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:34.834783 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4c804c-da64-4528-bf8a-d20753d1af86 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:34.835385 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4c804c-da64-4528-bf8a-d20753d1af86 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:34.837522 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4c804c-da64-4528-bf8a-d20753d1af86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:34.839204 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4c804c-da64-4528-bf8a-d20753d1af86 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:34.841491 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 144/278] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 15:59:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:34.846911 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:34.848448 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4c804c-da64-4528-bf8a-d20753d1af86 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:34.880249 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f3f4a7-843c-48e0-b24b-0e6f930da059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:34.881373 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3f4a7-843c-48e0-b24b-0e6f930da059 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:34.881829 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3f4a7-843c-48e0-b24b-0e6f930da059 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:34.882408 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3f4a7-843c-48e0-b24b-0e6f930da059 None admin] PATH_INFO: `/v3/users/38b1ecab4bf143bbbab1e50df71d53a5` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:34.899259 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f3f4a7-843c-48e0-b24b-0e6f930da059 None admin] RBAC: Authorizing `identity:get_user(user_id=38b1ecab4bf143bbbab1e50df71d53a5)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:34.901039 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f3f4a7-843c-48e0-b24b-0e6f930da059 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:34.903336 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 135/279] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 19 15:59:34 2022] GET /identity/v3/users/38b1ecab4bf143bbbab1e50df71d53a5 => generated 293 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 15:59:34.908624 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f3f4a7-843c-48e0-b24b-0e6f930da059 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:34.968903 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4c804c-da64-4528-bf8a-d20753d1af86 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:35.008390 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e800bb89-da38-49a4-8c18-b8c1a858b57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:35.009011 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e800bb89-da38-49a4-8c18-b8c1a858b57b None admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:35.009285 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e800bb89-da38-49a4-8c18-b8c1a858b57b None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:35.009562 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e800bb89-da38-49a4-8c18-b8c1a858b57b None admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:35.010218 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e800bb89-da38-49a4-8c18-b8c1a858b57b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:35.011695 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e800bb89-da38-49a4-8c18-b8c1a858b57b None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:35.021645 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f3f4a7-843c-48e0-b24b-0e6f930da059 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:35.035605 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 145/280] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Sep 19 15:59:34 2022] POST /identity/v3/users => generated 293 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 15:59:35.036875 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e800bb89-da38-49a4-8c18-b8c1a858b57b None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:35.063158 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:35.086642 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4436422-dbdb-49cb-847c-db443b41b2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:35.087874 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4436422-dbdb-49cb-847c-db443b41b2b7 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:35.089381 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4436422-dbdb-49cb-847c-db443b41b2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:35.089942 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4436422-dbdb-49cb-847c-db443b41b2b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:35.091010 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4436422-dbdb-49cb-847c-db443b41b2b7 None admin] RBAC: Adding query filter params (role.id=31b0628699ea4f98824f20e7568748cc, user.id=38b1ecab4bf143bbbab1e50df71d53a5, scope.system=all) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:35.091553 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4436422-dbdb-49cb-847c-db443b41b2b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:35.093767 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4436422-dbdb-49cb-847c-db443b41b2b7 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:35.102524 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 136/281] 10.222.0.57 () {56 vars in 1308 bytes} [Mon Sep 19 15:59:34 2022] GET /identity/v3/role_assignments?user.id=38b1ecab4bf143bbbab1e50df71d53a5&scope.system=all&role.id=31b0628699ea4f98824f20e7568748cc => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:35.178989 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e800bb89-da38-49a4-8c18-b8c1a858b57b None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:35.224644 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fd31df-57c2-436c-abbc-471837d1005c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:35.225608 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd31df-57c2-436c-abbc-471837d1005c None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:35.226111 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd31df-57c2-436c-abbc-471837d1005c None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:35.226584 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd31df-57c2-436c-abbc-471837d1005c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:35.240590 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fd31df-57c2-436c-abbc-471837d1005c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 15:59:35.246567 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-62fd31df-57c2-436c-abbc-471837d1005c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 15:59:35.246567 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 15:59:35.246567 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 15:59:35.246567 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 15:59:35.246567 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 15:59:35.246567 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 15:59:35.246567 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 15:59:35.246567 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 15:59:35.246567 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 15:59:35.246567 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 15:59:35.246567 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 15:59:35.246567 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 15:59:35.246567 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 15:59:35.246567 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 15:59:35.246567 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 15:59:35.246567 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 15:59:35.247496 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 15:59:35.247496 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 15:59:35.247496 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 15:59:35.247496 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 15:59:35.247496 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 15:59:35.247496 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 15:59:35.247496 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 15:59:35.247496 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 15:59:35.247496 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 15:59:35.247496 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 15:59:35.247496 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 15:59:35.247496 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 15:59:35.247496 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 15:59:35.247496 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 15:59:35.247496 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 15:59:35.247496 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 15:59:35.248294 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 15:59:35.248294 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 15:59:35.248294 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 15:59:35.248294 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 15:59:35.248294 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 15:59:35.248294 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 15:59:35.248294 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 15:59:35.248294 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 15:59:35.248294 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 15:59:35.248294 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 15:59:35.248294 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 15:59:35.248294 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 15:59:35.251267 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 146/282] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 15:59:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 15:59:35.257644 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4436422-dbdb-49cb-847c-db443b41b2b7 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:35.268364 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:35.381393 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4436422-dbdb-49cb-847c-db443b41b2b7 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:35.420250 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5dabef-a8ab-4eef-b660-61f2b13d84ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:35.421466 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5dabef-a8ab-4eef-b660-61f2b13d84ee None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:35.421742 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5dabef-a8ab-4eef-b660-61f2b13d84ee None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:35.422012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5dabef-a8ab-4eef-b660-61f2b13d84ee None admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:35.422705 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5dabef-a8ab-4eef-b660-61f2b13d84ee None admin] RBAC: Adding query filter params (name=reader) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:35.423010 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5dabef-a8ab-4eef-b660-61f2b13d84ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:35.424395 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5dabef-a8ab-4eef-b660-61f2b13d84ee None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:35.433529 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 137/283] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Sep 19 15:59:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:35.439120 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fd31df-57c2-436c-abbc-471837d1005c None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:35.536266 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:35.565261 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fd31df-57c2-436c-abbc-471837d1005c None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:35.603949 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740916fa-226c-4862-9a26-c8f457361208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:35.604643 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740916fa-226c-4862-9a26-c8f457361208 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:35.604890 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740916fa-226c-4862-9a26-c8f457361208 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:35.605161 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740916fa-226c-4862-9a26-c8f457361208 None admin] PATH_INFO: `/v3/users/ef6670387b9242ac857743e91aadc2f9` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:35.606346 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740916fa-226c-4862-9a26-c8f457361208 None admin] RBAC: Authorizing `identity:get_user(user_id=ef6670387b9242ac857743e91aadc2f9)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:35.607931 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740916fa-226c-4862-9a26-c8f457361208 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:35.609508 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e821a6-d206-4df7-81f3-8fc103329d75 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:35.609946 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 147/284] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 19 15:59:35 2022] GET /identity/v3/users/ef6670387b9242ac857743e91aadc2f9 => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:35.610020 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e821a6-d206-4df7-81f3-8fc103329d75 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:35.610102 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e821a6-d206-4df7-81f3-8fc103329d75 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:35.611533 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 138/285] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:59:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 15:59:35.615753 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740916fa-226c-4862-9a26-c8f457361208 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:35.617090 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2b5489-3e7c-45f7-b4b3-21cf20781565 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:35.617379 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2b5489-3e7c-45f7-b4b3-21cf20781565 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:35.617651 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2b5489-3e7c-45f7-b4b3-21cf20781565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:35.644647 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-8a2b5489-3e7c-45f7-b4b3-21cf20781565 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:59:35.646446 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-8a2b5489-3e7c-45f7-b4b3-21cf20781565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:59:35.718233 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:35.724949 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 139/286] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:59:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 15:59:35.746020 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740916fa-226c-4862-9a26-c8f457361208 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:35.781976 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6748041-a1f8-42aa-a1a2-0c87cf098792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:35.782638 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6748041-a1f8-42aa-a1a2-0c87cf098792 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:35.782916 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6748041-a1f8-42aa-a1a2-0c87cf098792 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:35.783185 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6748041-a1f8-42aa-a1a2-0c87cf098792 None admin] PATH_INFO: `/v3/projects/93b0db50dd5e4d3195beded37dd4105f` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:35.784322 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6748041-a1f8-42aa-a1a2-0c87cf098792 None admin] RBAC: Authorizing `identity:get_project(project_id=93b0db50dd5e4d3195beded37dd4105f)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:35.785763 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6748041-a1f8-42aa-a1a2-0c87cf098792 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:35.787494 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 148/287] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 19 15:59:35 2022] GET /identity/v3/projects/93b0db50dd5e4d3195beded37dd4105f => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:35.796253 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2b5489-3e7c-45f7-b4b3-21cf20781565 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:35.929337 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2b5489-3e7c-45f7-b4b3-21cf20781565 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:35.986837 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d84548e-e3a0-49bd-bdf8-e4ce96e60122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:35.988374 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d84548e-e3a0-49bd-bdf8-e4ce96e60122 None admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:35.989128 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d84548e-e3a0-49bd-bdf8-e4ce96e60122 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:35.989869 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d84548e-e3a0-49bd-bdf8-e4ce96e60122 None admin] PATH_INFO: `/v3/projects/93b0db50dd5e4d3195beded37dd4105f/users/ef6670387b9242ac857743e91aadc2f9/roles/625d911d4d934c1e80f0a57620077d49` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:35.993427 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d84548e-e3a0-49bd-bdf8-e4ce96e60122 None admin] RBAC: Authorizing `identity:create_grant(project_id=93b0db50dd5e4d3195beded37dd4105f, user_id=ef6670387b9242ac857743e91aadc2f9, role_id=625d911d4d934c1e80f0a57620077d49)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:35.996478 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d84548e-e3a0-49bd-bdf8-e4ce96e60122 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:36.030099 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 140/288] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Sep 19 15:59:35 2022] PUT /identity/v3/projects/93b0db50dd5e4d3195beded37dd4105f/users/ef6670387b9242ac857743e91aadc2f9/roles/625d911d4d934c1e80f0a57620077d49 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:59:36.222090 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6748041-a1f8-42aa-a1a2-0c87cf098792 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:36.251165 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:36.425386 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6748041-a1f8-42aa-a1a2-0c87cf098792 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:36.505804 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aedd42-e166-4093-9a03-70055666b784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:36.507236 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aedd42-e166-4093-9a03-70055666b784 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:36.507990 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aedd42-e166-4093-9a03-70055666b784 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:36.508725 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aedd42-e166-4093-9a03-70055666b784 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:36.528267 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aedd42-e166-4093-9a03-70055666b784 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 15:59:36.541511 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-77aedd42-e166-4093-9a03-70055666b784 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 15:59:36.541511 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 15:59:36.541511 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 15:59:36.541511 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 15:59:36.541511 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 15:59:36.541511 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 15:59:36.541511 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 15:59:36.541511 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 15:59:36.541511 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 15:59:36.541511 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 15:59:36.541511 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 15:59:36.541511 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 15:59:36.541511 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 15:59:36.541511 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 15:59:36.541511 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 15:59:36.541511 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 15:59:36.543260 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 15:59:36.543260 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 15:59:36.543260 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 15:59:36.543260 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 15:59:36.543260 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 15:59:36.543260 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 15:59:36.543260 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 15:59:36.543260 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 15:59:36.543260 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 15:59:36.543260 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 15:59:36.543260 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 15:59:36.543260 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 15:59:36.543260 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 15:59:36.543260 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 15:59:36.543260 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 15:59:36.543260 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 15:59:36.544813 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 15:59:36.544813 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 15:59:36.544813 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 15:59:36.544813 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 15:59:36.544813 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 15:59:36.544813 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 15:59:36.544813 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 15:59:36.544813 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 15:59:36.544813 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 15:59:36.544813 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 15:59:36.544813 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 15:59:36.544813 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 15:59:36.549323 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 149/289] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 15:59:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 15:59:36.557811 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d84548e-e3a0-49bd-bdf8-e4ce96e60122 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:36.673240 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:36.733235 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d84548e-e3a0-49bd-bdf8-e4ce96e60122 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:36.804098 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57603760-b7b1-451c-9aca-da8bb9dc7571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:36.804724 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57603760-b7b1-451c-9aca-da8bb9dc7571 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:36.805001 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57603760-b7b1-451c-9aca-da8bb9dc7571 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:36.805275 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57603760-b7b1-451c-9aca-da8bb9dc7571 None admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:36.805942 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57603760-b7b1-451c-9aca-da8bb9dc7571 None admin] RBAC: Adding query filter params (name=member) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:36.806250 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57603760-b7b1-451c-9aca-da8bb9dc7571 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:36.807636 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57603760-b7b1-451c-9aca-da8bb9dc7571 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:36.828808 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 141/290] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Sep 19 15:59:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:36.837587 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aedd42-e166-4093-9a03-70055666b784 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:37.006416 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:37.057263 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aedd42-e166-4093-9a03-70055666b784 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:37.132642 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67de49af-445f-4936-a6ef-478d605276c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:37.134120 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67de49af-445f-4936-a6ef-478d605276c2 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:37.134936 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67de49af-445f-4936-a6ef-478d605276c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:37.135696 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67de49af-445f-4936-a6ef-478d605276c2 None admin] PATH_INFO: `/v3/users/5eeb8545f33949879d25acc9fbb7c612` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:37.137825 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67de49af-445f-4936-a6ef-478d605276c2 None admin] RBAC: Authorizing `identity:get_user(user_id=5eeb8545f33949879d25acc9fbb7c612)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:37.147089 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67de49af-445f-4936-a6ef-478d605276c2 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:37.155365 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 150/291] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 19 15:59:36 2022] GET /identity/v3/users/5eeb8545f33949879d25acc9fbb7c612 => generated 297 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:37.159185 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57603760-b7b1-451c-9aca-da8bb9dc7571 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:37.332627 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57603760-b7b1-451c-9aca-da8bb9dc7571 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:37.356956 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:37.380067 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86c730e-d52c-45b1-a49c-1a48dd741b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:37.380723 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c730e-d52c-45b1-a49c-1a48dd741b78 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:37.381006 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c730e-d52c-45b1-a49c-1a48dd741b78 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:37.381324 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c730e-d52c-45b1-a49c-1a48dd741b78 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:37.464297 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86c730e-d52c-45b1-a49c-1a48dd741b78 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:37.465736 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86c730e-d52c-45b1-a49c-1a48dd741b78 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:37.647000 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 142/292] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 19 15:59:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 15:59:37.655544 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67de49af-445f-4936-a6ef-478d605276c2 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:37.803625 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67de49af-445f-4936-a6ef-478d605276c2 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:37.840585 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4cac14-b2fd-4931-aac8-22054b33b406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:37.841199 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4cac14-b2fd-4931-aac8-22054b33b406 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:37.841468 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4cac14-b2fd-4931-aac8-22054b33b406 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:37.841736 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4cac14-b2fd-4931-aac8-22054b33b406 None admin] PATH_INFO: `/v3/projects/93b0db50dd5e4d3195beded37dd4105f` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:37.843416 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4cac14-b2fd-4931-aac8-22054b33b406 None admin] RBAC: Authorizing `identity:get_project(project_id=93b0db50dd5e4d3195beded37dd4105f)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:37.844840 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4cac14-b2fd-4931-aac8-22054b33b406 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:37.846775 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 151/293] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 19 15:59:37 2022] GET /identity/v3/projects/93b0db50dd5e4d3195beded37dd4105f => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:37.852413 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86c730e-d52c-45b1-a49c-1a48dd741b78 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:37.868779 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d1bf9-05cf-459c-89f4-be6558274e52 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:37.869056 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d1bf9-05cf-459c-89f4-be6558274e52 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:37.869325 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d1bf9-05cf-459c-89f4-be6558274e52 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:37.870792 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 152/294] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:59:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 15:59:37.877444 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e5e7b-ed0f-4dbc-a21a-882430991310 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:37.877721 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e5e7b-ed0f-4dbc-a21a-882430991310 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:37.877993 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e5e7b-ed0f-4dbc-a21a-882430991310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:37.885908 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:37.913503 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-da5e5e7b-ed0f-4dbc-a21a-882430991310 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:59:37.915857 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-da5e5e7b-ed0f-4dbc-a21a-882430991310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:59:37.923753 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:38.030668 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 153/295] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:59:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 15:59:38.062096 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86c730e-d52c-45b1-a49c-1a48dd741b78 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:38.113784 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d18405b-6437-4537-a7c7-242aebd10dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:38.114600 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d18405b-6437-4537-a7c7-242aebd10dd7 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:38.114878 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d18405b-6437-4537-a7c7-242aebd10dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:38.115146 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d18405b-6437-4537-a7c7-242aebd10dd7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:38.115925 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d18405b-6437-4537-a7c7-242aebd10dd7 None admin] RBAC: Adding query filter params (role.id=31b0628699ea4f98824f20e7568748cc, scope.project.id=93b0db50dd5e4d3195beded37dd4105f, user.id=5eeb8545f33949879d25acc9fbb7c612) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:38.116234 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d18405b-6437-4537-a7c7-242aebd10dd7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:38.117646 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d18405b-6437-4537-a7c7-242aebd10dd7 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:38.124443 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 143/296] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 19 15:59:37 2022] GET /identity/v3/role_assignments?user.id=5eeb8545f33949879d25acc9fbb7c612&scope.project.id=93b0db50dd5e4d3195beded37dd4105f&role.id=31b0628699ea4f98824f20e7568748cc => generated 608 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:38.339837 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e5e7b-ed0f-4dbc-a21a-882430991310 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:38.455968 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e5e7b-ed0f-4dbc-a21a-882430991310 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:38.499628 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3f3568-708a-42b9-a268-a2c6ede02cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:38.500294 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f3568-708a-42b9-a268-a2c6ede02cb1 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:38.500574 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f3568-708a-42b9-a268-a2c6ede02cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:38.500850 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f3568-708a-42b9-a268-a2c6ede02cb1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:38.510222 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:38.515331 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3f3568-708a-42b9-a268-a2c6ede02cb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 15:59:38.520672 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-5d3f3568-708a-42b9-a268-a2c6ede02cb1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 15:59:38.520672 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 15:59:38.520672 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 15:59:38.520672 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 15:59:38.520672 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 15:59:38.520672 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 15:59:38.520672 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 15:59:38.520672 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 15:59:38.520672 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 15:59:38.520672 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 15:59:38.520672 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 15:59:38.520672 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 15:59:38.520672 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 15:59:38.520672 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 15:59:38.520672 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 15:59:38.520672 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 15:59:38.521607 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 15:59:38.521607 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 15:59:38.521607 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 15:59:38.521607 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 15:59:38.521607 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 15:59:38.521607 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 15:59:38.521607 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 15:59:38.521607 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 15:59:38.521607 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 15:59:38.521607 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 15:59:38.521607 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 15:59:38.521607 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 15:59:38.521607 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 15:59:38.521607 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 15:59:38.521607 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 15:59:38.521607 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 15:59:38.522364 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 15:59:38.522364 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 15:59:38.522364 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 15:59:38.522364 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 15:59:38.522364 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 15:59:38.522364 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 15:59:38.522364 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 15:59:38.522364 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 15:59:38.522364 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 15:59:38.522364 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 15:59:38.522364 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 15:59:38.522364 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 15:59:38.522364 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 154/297] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 15:59:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 15:59:38.529442 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d18405b-6437-4537-a7c7-242aebd10dd7 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:38.598970 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05da018f-1f18-4e67-93b6-ba58d77c2d8a None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:38.599576 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05da018f-1f18-4e67-93b6-ba58d77c2d8a None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:38.600050 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05da018f-1f18-4e67-93b6-ba58d77c2d8a None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:38.601775 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 155/298] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:59:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 15:59:38.608501 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da86b67-a91f-4caf-ab33-3f18768322a4 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:38.609213 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da86b67-a91f-4caf-ab33-3f18768322a4 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:38.609707 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da86b67-a91f-4caf-ab33-3f18768322a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:38.633808 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-9da86b67-a91f-4caf-ab33-3f18768322a4 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:59:38.635647 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-9da86b67-a91f-4caf-ab33-3f18768322a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:59:38.646340 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:38.668481 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d18405b-6437-4537-a7c7-242aebd10dd7 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:38.705450 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5835ac89-c1d0-494b-b32f-2604c9047ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:38.706070 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5835ac89-c1d0-494b-b32f-2604c9047ca4 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:38.706360 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5835ac89-c1d0-494b-b32f-2604c9047ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:38.706662 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5835ac89-c1d0-494b-b32f-2604c9047ca4 None admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:38.707335 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5835ac89-c1d0-494b-b32f-2604c9047ca4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:38.707806 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5835ac89-c1d0-494b-b32f-2604c9047ca4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:38.709394 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5835ac89-c1d0-494b-b32f-2604c9047ca4 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:38.712027 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 156/299] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:59:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 15:59:38.713310 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ac0f42-2800-4fbe-ac36-25781e07f6ae None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:38.713596 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ac0f42-2800-4fbe-ac36-25781e07f6ae None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:38.713867 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ac0f42-2800-4fbe-ac36-25781e07f6ae None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:38.715270 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 157/300] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:59:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 15:59:38.721886 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1765ed7-dcf6-4611-9b6e-98c862384191 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:38.722201 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1765ed7-dcf6-4611-9b6e-98c862384191 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:38.722469 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1765ed7-dcf6-4611-9b6e-98c862384191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:38.729529 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 144/301] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Sep 19 15:59:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:38.739497 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5835ac89-c1d0-494b-b32f-2604c9047ca4 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:38.745164 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-e1765ed7-dcf6-4611-9b6e-98c862384191 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:59:38.746416 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-e1765ed7-dcf6-4611-9b6e-98c862384191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:59:38.838409 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 158/302] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:59:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 15:59:38.875691 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5835ac89-c1d0-494b-b32f-2604c9047ca4 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:38.913053 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab581a49-3cbb-49bd-8799-4226a5a7ee21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:38.914070 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab581a49-3cbb-49bd-8799-4226a5a7ee21 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:38.914562 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab581a49-3cbb-49bd-8799-4226a5a7ee21 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:38.915032 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab581a49-3cbb-49bd-8799-4226a5a7ee21 None admin] PATH_INFO: `/v3/users/38b1ecab4bf143bbbab1e50df71d53a5` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:38.916687 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab581a49-3cbb-49bd-8799-4226a5a7ee21 None admin] RBAC: Authorizing `identity:get_user(user_id=38b1ecab4bf143bbbab1e50df71d53a5)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:38.918307 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab581a49-3cbb-49bd-8799-4226a5a7ee21 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:38.920563 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 145/303] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 19 15:59:38 2022] GET /identity/v3/users/38b1ecab4bf143bbbab1e50df71d53a5 => generated 293 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:38.926389 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1765ed7-dcf6-4611-9b6e-98c862384191 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:38.992771 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:39.008231 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab581a49-3cbb-49bd-8799-4226a5a7ee21 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:39.028923 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:39.048322 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1765ed7-dcf6-4611-9b6e-98c862384191 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:39.102473 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ab2f01-c917-4559-a80a-ce540ccc2c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:39.103193 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab2f01-c917-4559-a80a-ce540ccc2c17 None admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:39.103472 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab2f01-c917-4559-a80a-ce540ccc2c17 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:39.103746 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab2f01-c917-4559-a80a-ce540ccc2c17 None admin] PATH_INFO: `/v3/system/users/38b1ecab4bf143bbbab1e50df71d53a5/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:39.107200 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ab2f01-c917-4559-a80a-ce540ccc2c17 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=38b1ecab4bf143bbbab1e50df71d53a5, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:39.109032 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ab2f01-c917-4559-a80a-ce540ccc2c17 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:39.122125 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 159/304] 10.222.0.57 () {58 vars in 1334 bytes} [Mon Sep 19 15:59:38 2022] PUT /identity/v3/system/users/38b1ecab4bf143bbbab1e50df71d53a5/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:59:39.134842 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ab2f01-c917-4559-a80a-ce540ccc2c17 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:39.151056 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab581a49-3cbb-49bd-8799-4226a5a7ee21 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:39.199864 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95c981d-7e20-485b-a387-bb90373b0db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:39.200517 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c981d-7e20-485b-a387-bb90373b0db6 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:39.200847 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c981d-7e20-485b-a387-bb90373b0db6 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:39.201121 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c981d-7e20-485b-a387-bb90373b0db6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:39.213987 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95c981d-7e20-485b-a387-bb90373b0db6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 15:59:39.220225 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-d95c981d-7e20-485b-a387-bb90373b0db6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 15:59:39.220225 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 15:59:39.220225 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 15:59:39.220225 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 15:59:39.220225 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 15:59:39.220225 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 15:59:39.220225 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 15:59:39.220225 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 15:59:39.220225 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 15:59:39.220225 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 15:59:39.220225 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 15:59:39.220225 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 15:59:39.220225 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 15:59:39.220225 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 15:59:39.220225 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 15:59:39.220225 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 15:59:39.221183 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 15:59:39.221183 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 15:59:39.221183 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 15:59:39.221183 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 15:59:39.221183 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 15:59:39.221183 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 15:59:39.221183 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 15:59:39.221183 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 15:59:39.221183 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 15:59:39.221183 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 15:59:39.221183 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 15:59:39.221183 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 15:59:39.221183 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 15:59:39.221183 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 15:59:39.221183 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 15:59:39.221183 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 15:59:39.221950 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 15:59:39.221950 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 15:59:39.221950 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 15:59:39.221950 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 15:59:39.221950 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 15:59:39.221950 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 15:59:39.221950 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 15:59:39.221950 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 15:59:39.221950 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 15:59:39.221950 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 15:59:39.221950 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 15:59:39.221950 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 15:59:39.221950 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 146/305] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 15:59:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 15:59:39.229453 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95c981d-7e20-485b-a387-bb90373b0db6 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:39.295235 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ab2f01-c917-4559-a80a-ce540ccc2c17 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:39.321037 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:39.335224 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1417b565-3faa-4c0e-ae70-6413ad36a983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:39.335855 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1417b565-3faa-4c0e-ae70-6413ad36a983 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:39.336304 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:39.336604 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1417b565-3faa-4c0e-ae70-6413ad36a983 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:39.336882 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1417b565-3faa-4c0e-ae70-6413ad36a983 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:39.338115 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1417b565-3faa-4c0e-ae70-6413ad36a983 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:39.340044 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1417b565-3faa-4c0e-ae70-6413ad36a983 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:39.342302 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 160/306] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 15:59:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:39.343500 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422925e-d000-4f8a-86a4-02b42d68b51c None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:39.343777 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422925e-d000-4f8a-86a4-02b42d68b51c None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:39.344044 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422925e-d000-4f8a-86a4-02b42d68b51c None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:39.345393 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 161/307] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:59:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 15:59:39.350056 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b422925e-d000-4f8a-86a4-02b42d68b51c None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:39.369554 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95c981d-7e20-485b-a387-bb90373b0db6 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:39.404868 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27ea1e4-8ddf-433b-88a3-e30e83c0960a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:39.405389 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27ea1e4-8ddf-433b-88a3-e30e83c0960a None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:39.405745 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27ea1e4-8ddf-433b-88a3-e30e83c0960a None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:39.405980 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27ea1e4-8ddf-433b-88a3-e30e83c0960a None admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:39.406606 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27ea1e4-8ddf-433b-88a3-e30e83c0960a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:39.406907 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27ea1e4-8ddf-433b-88a3-e30e83c0960a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:39.408261 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27ea1e4-8ddf-433b-88a3-e30e83c0960a None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:39.417164 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 147/308] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Sep 19 15:59:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 15:59:39.419453 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68fb361-ea1e-4d6d-bf1e-f79a75909462 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:39.419942 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68fb361-ea1e-4d6d-bf1e-f79a75909462 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:39.421024 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68fb361-ea1e-4d6d-bf1e-f79a75909462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:39.440467 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-e68fb361-ea1e-4d6d-bf1e-f79a75909462 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:59:39.441639 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-e68fb361-ea1e-4d6d-bf1e-f79a75909462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:59:39.472973 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b422925e-d000-4f8a-86a4-02b42d68b51c None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:39.513443 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 148/309] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:59:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 15:59:39.513750 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2570185-7058-4bf4-a4c5-82f455846f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:39.514225 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68fb361-ea1e-4d6d-bf1e-f79a75909462 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:39.514567 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2570185-7058-4bf4-a4c5-82f455846f01 None admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:39.514845 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2570185-7058-4bf4-a4c5-82f455846f01 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:39.515112 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2570185-7058-4bf4-a4c5-82f455846f01 None admin] PATH_INFO: `/v3/groups` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:39.515738 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2570185-7058-4bf4-a4c5-82f455846f01 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:39.517179 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2570185-7058-4bf4-a4c5-82f455846f01 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:39.546536 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 162/310] 10.222.0.57 () {60 vars in 1134 bytes} [Mon Sep 19 15:59:39 2022] POST /identity/v3/groups => generated 229 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 15:59:39.600778 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:39.642708 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68fb361-ea1e-4d6d-bf1e-f79a75909462 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:39.691100 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b08522-c2b5-4ff7-b117-7da5009a685c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:39.692102 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b08522-c2b5-4ff7-b117-7da5009a685c None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:39.692669 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b08522-c2b5-4ff7-b117-7da5009a685c None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:39.693194 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b08522-c2b5-4ff7-b117-7da5009a685c None admin] PATH_INFO: `/v3/users/1f336d28c20449c7b1ea7375dcdbd196` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:39.711878 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b08522-c2b5-4ff7-b117-7da5009a685c None admin] RBAC: Authorizing `identity:get_user(user_id=1f336d28c20449c7b1ea7375dcdbd196)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:39.714788 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b08522-c2b5-4ff7-b117-7da5009a685c None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:39.723179 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 149/311] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 19 15:59:39 2022] GET /identity/v3/users/1f336d28c20449c7b1ea7375dcdbd196 => generated 293 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:39.725575 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2570185-7058-4bf4-a4c5-82f455846f01 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:39.782288 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b08522-c2b5-4ff7-b117-7da5009a685c None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:39.785919 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:39.848302 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2570185-7058-4bf4-a4c5-82f455846f01 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:39.884426 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb93bee-7d9d-4f90-a6bd-a3322f64a759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:39.885483 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb93bee-7d9d-4f90-a6bd-a3322f64a759 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:39.885961 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb93bee-7d9d-4f90-a6bd-a3322f64a759 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:39.886462 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb93bee-7d9d-4f90-a6bd-a3322f64a759 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:39.887407 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb93bee-7d9d-4f90-a6bd-a3322f64a759 None admin] RBAC: Adding query filter params (role.id=625d911d4d934c1e80f0a57620077d49, user.id=1f336d28c20449c7b1ea7375dcdbd196, scope.system=all) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:39.887960 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb93bee-7d9d-4f90-a6bd-a3322f64a759 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:39.890254 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb93bee-7d9d-4f90-a6bd-a3322f64a759 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:39.896023 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 163/312] 10.222.0.57 () {56 vars in 1308 bytes} [Mon Sep 19 15:59:39 2022] GET /identity/v3/role_assignments?user.id=1f336d28c20449c7b1ea7375dcdbd196&scope.system=all&role.id=625d911d4d934c1e80f0a57620077d49 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:39.915782 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b08522-c2b5-4ff7-b117-7da5009a685c None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:39.944781 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:39.957661 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fbdd84-b30e-4316-81a1-c6aa40b65bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:39.958306 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fbdd84-b30e-4316-81a1-c6aa40b65bc2 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:39.958616 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fbdd84-b30e-4316-81a1-c6aa40b65bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:39.958887 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fbdd84-b30e-4316-81a1-c6aa40b65bc2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:39.971997 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fbdd84-b30e-4316-81a1-c6aa40b65bc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 15:59:39.977027 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-b4fbdd84-b30e-4316-81a1-c6aa40b65bc2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 15:59:39.977027 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 15:59:39.977027 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 15:59:39.977027 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 15:59:39.977027 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 15:59:39.977027 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 15:59:39.977027 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 15:59:39.977027 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 15:59:39.977027 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 15:59:39.977027 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 15:59:39.977027 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 15:59:39.977027 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 15:59:39.977027 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 15:59:39.977027 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 15:59:39.977027 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 15:59:39.977027 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 15:59:39.977954 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 15:59:39.977954 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 15:59:39.977954 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 15:59:39.977954 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 15:59:39.977954 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 15:59:39.977954 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 15:59:39.977954 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 15:59:39.977954 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 15:59:39.977954 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 15:59:39.977954 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 15:59:39.977954 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 15:59:39.977954 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 15:59:39.977954 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 15:59:39.977954 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 15:59:39.977954 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 15:59:39.977954 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 15:59:39.978787 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 15:59:39.978787 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 15:59:39.978787 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 15:59:39.978787 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 15:59:39.978787 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 15:59:39.978787 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 15:59:39.978787 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 15:59:39.978787 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 15:59:39.978787 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 15:59:39.978787 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 15:59:39.978787 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 15:59:39.978787 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 15:59:39.978787 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 150/313] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 15:59:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 15:59:39.985917 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb93bee-7d9d-4f90-a6bd-a3322f64a759 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:40.117416 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb93bee-7d9d-4f90-a6bd-a3322f64a759 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:40.133139 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:40.154694 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82999ff-c3ed-42c4-8c68-6ee3fdacd9b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:40.155303 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82999ff-c3ed-42c4-8c68-6ee3fdacd9b9 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:40.155578 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82999ff-c3ed-42c4-8c68-6ee3fdacd9b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:40.155847 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82999ff-c3ed-42c4-8c68-6ee3fdacd9b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:40.156502 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82999ff-c3ed-42c4-8c68-6ee3fdacd9b9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:40.161474 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82999ff-c3ed-42c4-8c68-6ee3fdacd9b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:40.161474 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82999ff-c3ed-42c4-8c68-6ee3fdacd9b9 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:40.167922 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 164/314] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Sep 19 15:59:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:40.174303 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fbdd84-b30e-4316-81a1-c6aa40b65bc2 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:40.281748 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fbdd84-b30e-4316-81a1-c6aa40b65bc2 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:40.318143 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6649955-9235-41c3-b9d9-01d387d8075e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:40.318818 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6649955-9235-41c3-b9d9-01d387d8075e None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:40.319092 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6649955-9235-41c3-b9d9-01d387d8075e None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:40.319365 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6649955-9235-41c3-b9d9-01d387d8075e None admin] PATH_INFO: `/v3/users/ef6670387b9242ac857743e91aadc2f9` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:40.320517 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6649955-9235-41c3-b9d9-01d387d8075e None admin] RBAC: Authorizing `identity:get_user(user_id=ef6670387b9242ac857743e91aadc2f9)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:40.321955 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6649955-9235-41c3-b9d9-01d387d8075e None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:40.323878 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 151/315] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 19 15:59:40 2022] GET /identity/v3/users/ef6670387b9242ac857743e91aadc2f9 => generated 297 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:40.329974 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82999ff-c3ed-42c4-8c68-6ee3fdacd9b9 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:40.438704 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82999ff-c3ed-42c4-8c68-6ee3fdacd9b9 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:40.476827 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee30b16c-8004-4928-bc48-6f004ddf2fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:40.477441 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee30b16c-8004-4928-bc48-6f004ddf2fff None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:40.477713 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee30b16c-8004-4928-bc48-6f004ddf2fff None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:40.477982 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee30b16c-8004-4928-bc48-6f004ddf2fff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:40.524415 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:40.550530 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee30b16c-8004-4928-bc48-6f004ddf2fff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:40.552030 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee30b16c-8004-4928-bc48-6f004ddf2fff None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:40.660034 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 165/316] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 19 15:59:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 15:59:40.665469 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6649955-9235-41c3-b9d9-01d387d8075e None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:40.723541 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:40.782126 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6649955-9235-41c3-b9d9-01d387d8075e None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:40.816500 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224e5b43-94be-4fa1-ac53-611762b181ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:40.817048 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224e5b43-94be-4fa1-ac53-611762b181ff None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:40.817322 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224e5b43-94be-4fa1-ac53-611762b181ff None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:40.817588 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224e5b43-94be-4fa1-ac53-611762b181ff None admin] PATH_INFO: `/v3/projects/93b0db50dd5e4d3195beded37dd4105f` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:40.818729 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224e5b43-94be-4fa1-ac53-611762b181ff None admin] RBAC: Authorizing `identity:get_project(project_id=93b0db50dd5e4d3195beded37dd4105f)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:40.820153 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224e5b43-94be-4fa1-ac53-611762b181ff None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:40.821899 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 152/317] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 19 15:59:40 2022] GET /identity/v3/projects/93b0db50dd5e4d3195beded37dd4105f => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 15:59:40.828787 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee30b16c-8004-4928-bc48-6f004ddf2fff None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:40.915633 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:40.940405 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee30b16c-8004-4928-bc48-6f004ddf2fff None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:40.976411 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ba5c73-ce0d-4a15-bc42-43651286b88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:40.977020 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ba5c73-ce0d-4a15-bc42-43651286b88a None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:40.977296 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ba5c73-ce0d-4a15-bc42-43651286b88a None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:40.977568 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ba5c73-ce0d-4a15-bc42-43651286b88a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:40.978288 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ba5c73-ce0d-4a15-bc42-43651286b88a None admin] RBAC: Adding query filter params (role.id=625d911d4d934c1e80f0a57620077d49, scope.project.id=93b0db50dd5e4d3195beded37dd4105f, user.id=ef6670387b9242ac857743e91aadc2f9) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:40.978683 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ba5c73-ce0d-4a15-bc42-43651286b88a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:40.980100 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ba5c73-ce0d-4a15-bc42-43651286b88a None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:40.990833 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 166/318] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 19 15:59:40 2022] GET /identity/v3/role_assignments?user.id=ef6670387b9242ac857743e91aadc2f9&scope.project.id=93b0db50dd5e4d3195beded37dd4105f&role.id=625d911d4d934c1e80f0a57620077d49 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:41.400903 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ead4c3-a0a0-4c7d-8657-35fa453ef300 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:41.401415 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ead4c3-a0a0-4c7d-8657-35fa453ef300 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:41.401521 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ead4c3-a0a0-4c7d-8657-35fa453ef300 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:41.402869 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 153/319] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:59:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 15:59:41.408228 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9249e-ce4c-49a1-815e-22c3c68077bc None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:41.408521 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9249e-ce4c-49a1-815e-22c3c68077bc None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:41.408795 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9249e-ce4c-49a1-815e-22c3c68077bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:41.433650 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-3bd9249e-ce4c-49a1-815e-22c3c68077bc None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:59:41.434900 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-3bd9249e-ce4c-49a1-815e-22c3c68077bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:59:41.512696 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 167/320] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:59:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 15:59:41.778943 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ead4c3-a0a0-4c7d-8657-35fa453ef300 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:41.891970 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ead4c3-a0a0-4c7d-8657-35fa453ef300 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:41.913610 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26ede6c-8ccd-46f4-8f6e-8118e6b961e3 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:41.913914 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26ede6c-8ccd-46f4-8f6e-8118e6b961e3 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:41.914169 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26ede6c-8ccd-46f4-8f6e-8118e6b961e3 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:41.915741 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 168/321] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:59:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 15:59:41.916492 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:41.922864 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d97d9-89f7-4bfe-b2e3-561faded8334 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:41.923166 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d97d9-89f7-4bfe-b2e3-561faded8334 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:41.923472 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d97d9-89f7-4bfe-b2e3-561faded8334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:41.936119 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e02dfe-2bdf-4d93-8f39-81b4d62daf28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:41.936781 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e02dfe-2bdf-4d93-8f39-81b4d62daf28 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:41.937063 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e02dfe-2bdf-4d93-8f39-81b4d62daf28 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:41.937335 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e02dfe-2bdf-4d93-8f39-81b4d62daf28 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:41.942631 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-ed1d97d9-89f7-4bfe-b2e3-561faded8334 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:59:41.943877 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-ed1d97d9-89f7-4bfe-b2e3-561faded8334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:59:41.949891 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e02dfe-2bdf-4d93-8f39-81b4d62daf28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 15:59:41.954979 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-02e02dfe-2bdf-4d93-8f39-81b4d62daf28 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 15:59:41.954979 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 15:59:41.954979 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 15:59:41.954979 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 15:59:41.954979 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 15:59:41.954979 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 15:59:41.954979 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 15:59:41.954979 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 15:59:41.954979 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 15:59:41.954979 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 15:59:41.954979 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 15:59:41.954979 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 15:59:41.954979 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 15:59:41.954979 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 15:59:41.954979 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 15:59:41.954979 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 15:59:41.955913 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 15:59:41.955913 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 15:59:41.955913 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 15:59:41.955913 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 15:59:41.955913 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 15:59:41.955913 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 15:59:41.955913 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 15:59:41.955913 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 15:59:41.955913 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 15:59:41.955913 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 15:59:41.955913 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 15:59:41.955913 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 15:59:41.955913 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 15:59:41.955913 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 15:59:41.955913 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 15:59:41.955913 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 15:59:41.956769 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 15:59:41.956769 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 15:59:41.956769 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 15:59:41.956769 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 15:59:41.956769 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 15:59:41.956769 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 15:59:41.956769 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 15:59:41.956769 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 15:59:41.956769 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 15:59:41.956769 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 15:59:41.956769 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 15:59:41.956769 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 15:59:41.956769 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 154/322] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 15:59:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 15:59:41.962072 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e02dfe-2bdf-4d93-8f39-81b4d62daf28 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:42.019928 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:42.025141 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 169/323] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:59:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 15:59:42.090815 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e02dfe-2bdf-4d93-8f39-81b4d62daf28 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:42.126650 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f919fba8-a030-46b7-a10f-7b2861c235f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:42.127260 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f919fba8-a030-46b7-a10f-7b2861c235f6 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:42.127539 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f919fba8-a030-46b7-a10f-7b2861c235f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:42.127812 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f919fba8-a030-46b7-a10f-7b2861c235f6 None admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:42.128472 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f919fba8-a030-46b7-a10f-7b2861c235f6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:42.128827 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f919fba8-a030-46b7-a10f-7b2861c235f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:42.130189 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f919fba8-a030-46b7-a10f-7b2861c235f6 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:42.143224 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 155/324] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Sep 19 15:59:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 15:59:42.144520 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d97d9-89f7-4bfe-b2e3-561faded8334 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:42.184143 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0420f72b-8f25-4773-939e-3b63c50c235a None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:42.184537 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0420f72b-8f25-4773-939e-3b63c50c235a None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:42.184816 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0420f72b-8f25-4773-939e-3b63c50c235a None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:42.186322 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 156/325] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:59:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 15:59:42.191666 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae81e2c-71ba-40f7-bd32-d2d9906d7cca None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:42.191947 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae81e2c-71ba-40f7-bd32-d2d9906d7cca None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:42.192219 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae81e2c-71ba-40f7-bd32-d2d9906d7cca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:42.210587 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-4ae81e2c-71ba-40f7-bd32-d2d9906d7cca None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:59:42.211778 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-4ae81e2c-71ba-40f7-bd32-d2d9906d7cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:59:42.254443 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:42.261905 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d97d9-89f7-4bfe-b2e3-561faded8334 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:42.289587 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 157/326] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:59:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 15:59:42.298012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae81e2c-71ba-40f7-bd32-d2d9906d7cca None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:42.299160 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9889b9a7-caac-42c1-aa67-1840bfadd1a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:42.299798 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9889b9a7-caac-42c1-aa67-1840bfadd1a1 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:42.300076 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9889b9a7-caac-42c1-aa67-1840bfadd1a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:42.300457 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9889b9a7-caac-42c1-aa67-1840bfadd1a1 None admin] PATH_INFO: `/v3/users/38b1ecab4bf143bbbab1e50df71d53a5` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:42.301820 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9889b9a7-caac-42c1-aa67-1840bfadd1a1 None admin] RBAC: Authorizing `identity:get_user(user_id=38b1ecab4bf143bbbab1e50df71d53a5)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:42.303278 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9889b9a7-caac-42c1-aa67-1840bfadd1a1 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:42.305028 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 170/327] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 19 15:59:42 2022] GET /identity/v3/users/38b1ecab4bf143bbbab1e50df71d53a5 => generated 293 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:42.310865 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9889b9a7-caac-42c1-aa67-1840bfadd1a1 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:42.389876 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:42.406003 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae81e2c-71ba-40f7-bd32-d2d9906d7cca None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:42.425064 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9889b9a7-caac-42c1-aa67-1840bfadd1a1 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:42.440771 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db1f8fb-1a74-401f-b674-e3621fbe3709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:42.441397 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db1f8fb-1a74-401f-b674-e3621fbe3709 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:42.441715 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db1f8fb-1a74-401f-b674-e3621fbe3709 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:42.441981 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db1f8fb-1a74-401f-b674-e3621fbe3709 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:42.443134 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db1f8fb-1a74-401f-b674-e3621fbe3709 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:42.444539 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db1f8fb-1a74-401f-b674-e3621fbe3709 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:42.446161 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 158/328] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 15:59:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 15:59:42.452108 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db1f8fb-1a74-401f-b674-e3621fbe3709 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:42.462144 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc6704a-eabf-4382-8c90-df1ae2c7512b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:42.463177 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc6704a-eabf-4382-8c90-df1ae2c7512b None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:42.463462 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc6704a-eabf-4382-8c90-df1ae2c7512b None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:42.463735 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc6704a-eabf-4382-8c90-df1ae2c7512b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:42.464781 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc6704a-eabf-4382-8c90-df1ae2c7512b None admin] RBAC: Adding query filter params (role.id=31b0628699ea4f98824f20e7568748cc, user.id=38b1ecab4bf143bbbab1e50df71d53a5, scope.system=all) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:42.465184 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc6704a-eabf-4382-8c90-df1ae2c7512b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:42.466636 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc6704a-eabf-4382-8c90-df1ae2c7512b None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:42.479365 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 171/329] 10.222.0.57 () {56 vars in 1308 bytes} [Mon Sep 19 15:59:42 2022] GET /identity/v3/role_assignments?user.id=38b1ecab4bf143bbbab1e50df71d53a5&scope.system=all&role.id=31b0628699ea4f98824f20e7568748cc => generated 510 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 15:59:42.496057 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:42.562744 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc6704a-eabf-4382-8c90-df1ae2c7512b None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:42.563602 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db1f8fb-1a74-401f-b674-e3621fbe3709 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:42.605212 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75e3cc5-4957-4db1-ae6e-55dd61097aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:42.606013 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75e3cc5-4957-4db1-ae6e-55dd61097aa1 None admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:42.606730 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75e3cc5-4957-4db1-ae6e-55dd61097aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:42.607014 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75e3cc5-4957-4db1-ae6e-55dd61097aa1 None admin] PATH_INFO: `/v3/groups` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:42.607923 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75e3cc5-4957-4db1-ae6e-55dd61097aa1 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:42.609408 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75e3cc5-4957-4db1-ae6e-55dd61097aa1 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:42.631019 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 159/330] 10.222.0.57 () {60 vars in 1134 bytes} [Mon Sep 19 15:59:42 2022] POST /identity/v3/groups => generated 226 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 15:59:42.693887 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc6704a-eabf-4382-8c90-df1ae2c7512b None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:42.726100 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:42.737089 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62298e81-9b58-4c7e-b2cf-419e3e1735db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:42.738135 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62298e81-9b58-4c7e-b2cf-419e3e1735db None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:42.738441 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62298e81-9b58-4c7e-b2cf-419e3e1735db None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:42.738836 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62298e81-9b58-4c7e-b2cf-419e3e1735db None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:42.750776 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62298e81-9b58-4c7e-b2cf-419e3e1735db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 15:59:42.756245 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-62298e81-9b58-4c7e-b2cf-419e3e1735db None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 15:59:42.756245 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 15:59:42.756245 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 15:59:42.756245 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 15:59:42.756245 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 15:59:42.756245 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 15:59:42.756245 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 15:59:42.756245 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 15:59:42.756245 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 15:59:42.756245 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 15:59:42.756245 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 15:59:42.756245 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 15:59:42.756245 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 15:59:42.756245 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 15:59:42.756245 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 15:59:42.756245 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 15:59:42.757356 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 15:59:42.757356 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 15:59:42.757356 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 15:59:42.757356 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 15:59:42.757356 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 15:59:42.757356 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 15:59:42.757356 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 15:59:42.757356 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 15:59:42.757356 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 15:59:42.757356 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 15:59:42.757356 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 15:59:42.757356 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 15:59:42.757356 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 15:59:42.757356 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 15:59:42.757356 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 15:59:42.757356 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 15:59:42.758169 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 15:59:42.758169 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 15:59:42.758169 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 15:59:42.758169 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 15:59:42.758169 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 15:59:42.758169 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 15:59:42.758169 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 15:59:42.758169 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 15:59:42.758169 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 15:59:42.758169 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 15:59:42.758169 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 15:59:42.758169 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 15:59:42.758169 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 172/331] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 15:59:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 15:59:42.763760 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75e3cc5-4957-4db1-ae6e-55dd61097aa1 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:42.869623 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:42.873080 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75e3cc5-4957-4db1-ae6e-55dd61097aa1 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:42.911328 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05b689b-8020-4d62-8fae-6b377fd50030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:42.911942 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b689b-8020-4d62-8fae-6b377fd50030 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:42.912214 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b689b-8020-4d62-8fae-6b377fd50030 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:42.912485 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b689b-8020-4d62-8fae-6b377fd50030 None admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:42.913145 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05b689b-8020-4d62-8fae-6b377fd50030 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:42.913503 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05b689b-8020-4d62-8fae-6b377fd50030 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:42.914907 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05b689b-8020-4d62-8fae-6b377fd50030 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:42.924812 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 160/332] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Sep 19 15:59:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:42.930351 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62298e81-9b58-4c7e-b2cf-419e3e1735db None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:43.037225 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62298e81-9b58-4c7e-b2cf-419e3e1735db None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:43.075191 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cd6a5c-bdd3-498e-b3f1-113a93b36c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:43.075857 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd6a5c-bdd3-498e-b3f1-113a93b36c13 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:43.076136 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd6a5c-bdd3-498e-b3f1-113a93b36c13 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:43.076928 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd6a5c-bdd3-498e-b3f1-113a93b36c13 None admin] PATH_INFO: `/v3/users/1f336d28c20449c7b1ea7375dcdbd196` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:43.078217 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cd6a5c-bdd3-498e-b3f1-113a93b36c13 None admin] RBAC: Authorizing `identity:get_user(user_id=1f336d28c20449c7b1ea7375dcdbd196)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:43.079763 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cd6a5c-bdd3-498e-b3f1-113a93b36c13 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:43.081493 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 173/333] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 19 15:59:42 2022] GET /identity/v3/users/1f336d28c20449c7b1ea7375dcdbd196 => generated 293 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:43.087657 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05b689b-8020-4d62-8fae-6b377fd50030 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:43.226039 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05b689b-8020-4d62-8fae-6b377fd50030 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:43.264611 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f46beb-08af-44f4-b559-3ed0f2de65a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:43.265272 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f46beb-08af-44f4-b559-3ed0f2de65a9 None admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:43.265546 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f46beb-08af-44f4-b559-3ed0f2de65a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:43.265817 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f46beb-08af-44f4-b559-3ed0f2de65a9 None admin] PATH_INFO: `/v3/system/users/1f336d28c20449c7b1ea7375dcdbd196/roles/625d911d4d934c1e80f0a57620077d49` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:43.268071 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f46beb-08af-44f4-b559-3ed0f2de65a9 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=1f336d28c20449c7b1ea7375dcdbd196, role_id=625d911d4d934c1e80f0a57620077d49)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:43.269459 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f46beb-08af-44f4-b559-3ed0f2de65a9 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:43.283123 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 161/334] 10.222.0.57 () {58 vars in 1334 bytes} [Mon Sep 19 15:59:43 2022] PUT /identity/v3/system/users/1f336d28c20449c7b1ea7375dcdbd196/roles/625d911d4d934c1e80f0a57620077d49 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:59:45.081702 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e2877e-fa50-41f8-bf46-8a9903cc5366 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:45.082739 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e2877e-fa50-41f8-bf46-8a9903cc5366 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:45.083310 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e2877e-fa50-41f8-bf46-8a9903cc5366 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:45.085148 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 174/335] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:59:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 15:59:45.091037 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9fdd91-fce5-40b3-b8cb-990804c8d623 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:45.092476 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9fdd91-fce5-40b3-b8cb-990804c8d623 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:45.092993 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9fdd91-fce5-40b3-b8cb-990804c8d623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:45.112616 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-be9fdd91-fce5-40b3-b8cb-990804c8d623 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:59:45.114168 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-be9fdd91-fce5-40b3-b8cb-990804c8d623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:59:45.149387 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:45.212227 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 162/336] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:59:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 15:59:45.287964 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b11a32-5994-40cb-a6aa-4ec73ccc5150 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:45.288430 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b11a32-5994-40cb-a6aa-4ec73ccc5150 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:45.288542 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b11a32-5994-40cb-a6aa-4ec73ccc5150 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:45.290306 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 175/337] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:59:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 15:59:45.298077 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b7c5c3-f0bf-48a1-95ff-c201b37691e2 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:45.298430 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b7c5c3-f0bf-48a1-95ff-c201b37691e2 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:45.298641 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b7c5c3-f0bf-48a1-95ff-c201b37691e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:45.328390 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-a6b7c5c3-f0bf-48a1-95ff-c201b37691e2 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:59:45.329568 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-a6b7c5c3-f0bf-48a1-95ff-c201b37691e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:59:45.405389 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 163/338] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:59:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 15:59:45.792011 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b11a32-5994-40cb-a6aa-4ec73ccc5150 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:45.879199 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:45.945726 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b11a32-5994-40cb-a6aa-4ec73ccc5150 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:45.988857 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bedb497-23b7-4d87-9d73-5d55423392af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:45.989499 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bedb497-23b7-4d87-9d73-5d55423392af None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:45.989767 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bedb497-23b7-4d87-9d73-5d55423392af None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:45.990034 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bedb497-23b7-4d87-9d73-5d55423392af None admin] PATH_INFO: `/v3/roles/member` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:46.004134 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b7c5c3-f0bf-48a1-95ff-c201b37691e2 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:46.004316 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bedb497-23b7-4d87-9d73-5d55423392af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 15:59:46.009118 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-6bedb497-23b7-4d87-9d73-5d55423392af None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 15:59:46.009118 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 15:59:46.009118 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 15:59:46.009118 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 15:59:46.009118 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 15:59:46.009118 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 15:59:46.009118 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 15:59:46.009118 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 15:59:46.009118 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 15:59:46.009118 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 15:59:46.009118 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 15:59:46.009118 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 15:59:46.009118 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 15:59:46.009118 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 15:59:46.009118 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 15:59:46.009118 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 15:59:46.010066 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 15:59:46.010066 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 15:59:46.010066 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 15:59:46.010066 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 15:59:46.010066 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 15:59:46.010066 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 15:59:46.010066 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 15:59:46.010066 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 15:59:46.010066 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 15:59:46.010066 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 15:59:46.010066 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 15:59:46.010066 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 15:59:46.010066 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 15:59:46.010066 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 15:59:46.010066 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 15:59:46.010066 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 15:59:46.010912 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 15:59:46.010912 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 15:59:46.010912 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 15:59:46.010912 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 15:59:46.010912 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 15:59:46.010912 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 15:59:46.010912 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 15:59:46.010912 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 15:59:46.010912 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 15:59:46.010912 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 15:59:46.010912 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 15:59:46.010912 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 15:59:46.010912 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 176/339] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 15:59:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 15:59:46.017951 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bedb497-23b7-4d87-9d73-5d55423392af None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:46.115563 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:46.128339 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b7c5c3-f0bf-48a1-95ff-c201b37691e2 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:46.167054 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7211bce6-e070-4a6b-a000-330878cfb1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:46.167704 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7211bce6-e070-4a6b-a000-330878cfb1a6 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:46.167984 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7211bce6-e070-4a6b-a000-330878cfb1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:46.168259 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7211bce6-e070-4a6b-a000-330878cfb1a6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:46.182696 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7211bce6-e070-4a6b-a000-330878cfb1a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 15:59:46.194012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-7211bce6-e070-4a6b-a000-330878cfb1a6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 15:59:46.194012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 15:59:46.194012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 15:59:46.194012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 15:59:46.194012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 15:59:46.194012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 15:59:46.194012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 15:59:46.194012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 15:59:46.194012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 15:59:46.194012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 15:59:46.194012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 15:59:46.194012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 15:59:46.194012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 15:59:46.194012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 15:59:46.194012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 15:59:46.194012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 15:59:46.194987 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 15:59:46.194987 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 15:59:46.194987 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 15:59:46.194987 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 15:59:46.194987 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 15:59:46.194987 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 15:59:46.194987 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 15:59:46.194987 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 15:59:46.194987 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 15:59:46.194987 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 15:59:46.194987 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 15:59:46.194987 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 15:59:46.194987 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 15:59:46.194987 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 15:59:46.194987 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 15:59:46.194987 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 15:59:46.195752 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 15:59:46.195752 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 15:59:46.195752 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 15:59:46.195752 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 15:59:46.195752 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 15:59:46.195752 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 15:59:46.195752 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 15:59:46.195752 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 15:59:46.195752 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 15:59:46.195752 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 15:59:46.195752 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 15:59:46.195752 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 15:59:46.195752 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 164/340] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Sep 19 15:59:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 15:59:46.197381 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cf12e9-9546-4b7b-ae76-45cdd445b7c4 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:46.197669 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cf12e9-9546-4b7b-ae76-45cdd445b7c4 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:46.197947 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cf12e9-9546-4b7b-ae76-45cdd445b7c4 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:46.200108 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 165/341] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:59:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 15:59:46.204219 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cf12e9-9546-4b7b-ae76-45cdd445b7c4 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:46.274335 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bedb497-23b7-4d87-9d73-5d55423392af None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:46.302475 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:46.330532 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cf12e9-9546-4b7b-ae76-45cdd445b7c4 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:46.335917 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0608e106-660f-4d15-b1e1-70eedb726d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:46.336813 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0608e106-660f-4d15-b1e1-70eedb726d22 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:46.337300 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0608e106-660f-4d15-b1e1-70eedb726d22 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:46.337775 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0608e106-660f-4d15-b1e1-70eedb726d22 None admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:46.341158 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0608e106-660f-4d15-b1e1-70eedb726d22 None admin] RBAC: Adding query filter params (name=member) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:46.341708 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0608e106-660f-4d15-b1e1-70eedb726d22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:46.343917 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0608e106-660f-4d15-b1e1-70eedb726d22 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:46.359182 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 177/342] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Sep 19 15:59:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 15:59:46.361035 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8482310-ef6e-4093-93c6-16fe1649e8f0 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:46.361577 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8482310-ef6e-4093-93c6-16fe1649e8f0 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:46.362060 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8482310-ef6e-4093-93c6-16fe1649e8f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:46.379800 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978f5dcb-5a86-4bf9-afcd-c1c89fdfb2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:46.380368 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f5dcb-5a86-4bf9-afcd-c1c89fdfb2ac None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:46.380659 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f5dcb-5a86-4bf9-afcd-c1c89fdfb2ac None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:46.380936 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f5dcb-5a86-4bf9-afcd-c1c89fdfb2ac None admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:46.381633 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978f5dcb-5a86-4bf9-afcd-c1c89fdfb2ac None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:46.381941 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978f5dcb-5a86-4bf9-afcd-c1c89fdfb2ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:46.383411 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978f5dcb-5a86-4bf9-afcd-c1c89fdfb2ac None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:46.402211 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 166/343] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 19 15:59:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:46.403546 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978f5dcb-5a86-4bf9-afcd-c1c89fdfb2ac None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:46.403675 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-a8482310-ef6e-4093-93c6-16fe1649e8f0 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:59:46.409648 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-a8482310-ef6e-4093-93c6-16fe1649e8f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:59:46.425879 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:46.502514 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 178/344] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:59:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 15:59:46.503527 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8482310-ef6e-4093-93c6-16fe1649e8f0 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:46.566080 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978f5dcb-5a86-4bf9-afcd-c1c89fdfb2ac None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:46.611742 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1befc14e-68af-4dc3-b3c5-afc9958816a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:46.612362 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1befc14e-68af-4dc3-b3c5-afc9958816a3 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:46.612637 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1befc14e-68af-4dc3-b3c5-afc9958816a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:46.612905 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1befc14e-68af-4dc3-b3c5-afc9958816a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:46.668656 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8482310-ef6e-4093-93c6-16fe1649e8f0 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:46.682808 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:46.690822 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1befc14e-68af-4dc3-b3c5-afc9958816a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:46.692618 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1befc14e-68af-4dc3-b3c5-afc9958816a3 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:46.704114 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:46.735875 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d241827-88f5-4d69-86ea-473c9965a936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:46.736785 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d241827-88f5-4d69-86ea-473c9965a936 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:46.737223 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d241827-88f5-4d69-86ea-473c9965a936 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:46.737749 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d241827-88f5-4d69-86ea-473c9965a936 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:46.800796 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 167/345] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 19 15:59:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 15:59:46.802229 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c8c87b-6d88-4630-9af6-a233580d84aa None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:46.803708 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c8c87b-6d88-4630-9af6-a233580d84aa None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:46.804085 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c8c87b-6d88-4630-9af6-a233580d84aa None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:46.805817 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 168/346] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:59:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 15:59:46.806485 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c8c87b-6d88-4630-9af6-a233580d84aa None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:46.897807 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d241827-88f5-4d69-86ea-473c9965a936 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:46.899305 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d241827-88f5-4d69-86ea-473c9965a936 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:47.005047 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c8c87b-6d88-4630-9af6-a233580d84aa None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:47.018722 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:47.057352 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 179/347] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 19 15:59:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 554 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 15:59:47.058698 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2bcc2a-8c79-4fff-a19c-75669074ca6d None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:47.058981 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2bcc2a-8c79-4fff-a19c-75669074ca6d None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:47.059251 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2bcc2a-8c79-4fff-a19c-75669074ca6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:47.076783 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:47.083456 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-ab2bcc2a-8c79-4fff-a19c-75669074ca6d None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:59:47.084779 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-ab2bcc2a-8c79-4fff-a19c-75669074ca6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:59:47.087300 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08684dc-9deb-45be-bbda-cc539935c167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:47.088032 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08684dc-9deb-45be-bbda-cc539935c167 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:47.088221 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08684dc-9deb-45be-bbda-cc539935c167 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:47.088481 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08684dc-9deb-45be-bbda-cc539935c167 None admin] PATH_INFO: `/v3/projects/ad7bddf539af4f8dabada63995dab07f` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:47.090079 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08684dc-9deb-45be-bbda-cc539935c167 None admin] RBAC: Authorizing `identity:get_project(project_id=ad7bddf539af4f8dabada63995dab07f)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:47.091638 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08684dc-9deb-45be-bbda-cc539935c167 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:47.094056 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 169/348] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 19 15:59:46 2022] GET /identity/v3/projects/ad7bddf539af4f8dabada63995dab07f => generated 298 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 15:59:47.094681 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08684dc-9deb-45be-bbda-cc539935c167 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:47.161692 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 180/349] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:59:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 15:59:47.162597 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2bcc2a-8c79-4fff-a19c-75669074ca6d None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:47.225531 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08684dc-9deb-45be-bbda-cc539935c167 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:47.263589 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723e2e89-383f-4caa-b16f-32ef228c8a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:47.264708 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723e2e89-383f-4caa-b16f-32ef228c8a66 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:47.264996 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723e2e89-383f-4caa-b16f-32ef228c8a66 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:47.265244 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723e2e89-383f-4caa-b16f-32ef228c8a66 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:47.291566 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723e2e89-383f-4caa-b16f-32ef228c8a66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 15:59:47.299065 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2bcc2a-8c79-4fff-a19c-75669074ca6d None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:47.302888 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-723e2e89-383f-4caa-b16f-32ef228c8a66 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 15:59:47.302888 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 15:59:47.302888 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 15:59:47.302888 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 15:59:47.302888 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 15:59:47.302888 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 15:59:47.302888 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 15:59:47.302888 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 15:59:47.302888 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 15:59:47.302888 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 15:59:47.302888 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 15:59:47.302888 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 15:59:47.302888 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 15:59:47.302888 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 15:59:47.302888 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 15:59:47.302888 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 15:59:47.303861 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 15:59:47.303861 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 15:59:47.303861 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 15:59:47.303861 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 15:59:47.303861 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 15:59:47.303861 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 15:59:47.303861 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 15:59:47.303861 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 15:59:47.303861 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 15:59:47.303861 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 15:59:47.303861 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 15:59:47.303861 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 15:59:47.303861 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 15:59:47.303861 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 15:59:47.303861 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 15:59:47.303861 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 15:59:47.304915 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 15:59:47.304915 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 15:59:47.304915 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 15:59:47.304915 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 15:59:47.304915 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 15:59:47.304915 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 15:59:47.304915 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 15:59:47.304915 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 15:59:47.304915 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 15:59:47.304915 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 15:59:47.304915 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 15:59:47.304915 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 15:59:47.307760 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 170/350] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 15:59:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 15:59:47.313956 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9225f5bd-0f9f-4513-98fa-2ff77fc95db4 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:47.314648 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9225f5bd-0f9f-4513-98fa-2ff77fc95db4 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:47.315126 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9225f5bd-0f9f-4513-98fa-2ff77fc95db4 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:47.316808 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 171/351] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:59:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 15:59:47.317946 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9225f5bd-0f9f-4513-98fa-2ff77fc95db4 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:47.338790 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aa05b9-c779-42d5-9a39-78082b427151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:47.339475 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa05b9-c779-42d5-9a39-78082b427151 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:47.339751 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa05b9-c779-42d5-9a39-78082b427151 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:47.340004 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa05b9-c779-42d5-9a39-78082b427151 None admin] PATH_INFO: `/v3/projects/ad7bddf539af4f8dabada63995dab07f` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:47.341127 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aa05b9-c779-42d5-9a39-78082b427151 None admin] RBAC: Authorizing `identity:get_project(project_id=ad7bddf539af4f8dabada63995dab07f)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:47.342653 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aa05b9-c779-42d5-9a39-78082b427151 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:47.344482 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 181/352] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 19 15:59:47 2022] GET /identity/v3/projects/ad7bddf539af4f8dabada63995dab07f => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:47.345429 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c145cd3-ab57-4930-81ed-60c9d51bbd6b None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:47.345708 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c145cd3-ab57-4930-81ed-60c9d51bbd6b None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:47.345968 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c145cd3-ab57-4930-81ed-60c9d51bbd6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:47.353823 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:47.366505 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-7c145cd3-ab57-4930-81ed-60c9d51bbd6b None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:59:47.367716 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-7c145cd3-ab57-4930-81ed-60c9d51bbd6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:59:47.371433 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:47.443447 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 182/353] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:59:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 15:59:47.444644 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c145cd3-ab57-4930-81ed-60c9d51bbd6b None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:47.452323 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9225f5bd-0f9f-4513-98fa-2ff77fc95db4 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:47.492616 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0fa536-2fb9-4d94-b589-8b7ce0349b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:47.493792 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0fa536-2fb9-4d94-b589-8b7ce0349b26 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:47.494326 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0fa536-2fb9-4d94-b589-8b7ce0349b26 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:47.494848 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0fa536-2fb9-4d94-b589-8b7ce0349b26 None admin] PATH_INFO: `/v3/groups/92fde2624f7744c2b5e809954f75a8d9` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:47.501701 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0fa536-2fb9-4d94-b589-8b7ce0349b26 None admin] RBAC: Authorizing `identity:get_group(group_id=92fde2624f7744c2b5e809954f75a8d9)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:47.503522 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0fa536-2fb9-4d94-b589-8b7ce0349b26 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:47.505533 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 172/354] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Sep 19 15:59:47 2022] GET /identity/v3/groups/92fde2624f7744c2b5e809954f75a8d9 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:47.506770 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0fa536-2fb9-4d94-b589-8b7ce0349b26 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:47.579364 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c145cd3-ab57-4930-81ed-60c9d51bbd6b None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:47.621854 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:47.625608 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0fa536-2fb9-4d94-b589-8b7ce0349b26 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:47.626663 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabbacf0-3ee6-4da7-9c0c-62cd09591f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:47.627282 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabbacf0-3ee6-4da7-9c0c-62cd09591f38 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:47.627555 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabbacf0-3ee6-4da7-9c0c-62cd09591f38 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:47.627827 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabbacf0-3ee6-4da7-9c0c-62cd09591f38 None admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:47.628794 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabbacf0-3ee6-4da7-9c0c-62cd09591f38 None admin] RBAC: Adding query filter params (name=member) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:47.629287 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabbacf0-3ee6-4da7-9c0c-62cd09591f38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:47.630934 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabbacf0-3ee6-4da7-9c0c-62cd09591f38 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:47.632404 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:47.641193 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 183/355] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Sep 19 15:59:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:47.642041 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabbacf0-3ee6-4da7-9c0c-62cd09591f38 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:47.669366 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d53015-7f8a-4b68-a269-806e4e6b5b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:47.671010 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d53015-7f8a-4b68-a269-806e4e6b5b33 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:47.671523 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d53015-7f8a-4b68-a269-806e4e6b5b33 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:47.671994 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d53015-7f8a-4b68-a269-806e4e6b5b33 None admin] PATH_INFO: `/v3/groups/92fde2624f7744c2b5e809954f75a8d9` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:47.673575 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d53015-7f8a-4b68-a269-806e4e6b5b33 None admin] RBAC: Authorizing `identity:get_group(group_id=92fde2624f7744c2b5e809954f75a8d9)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:47.675363 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d53015-7f8a-4b68-a269-806e4e6b5b33 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:47.677536 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 173/356] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Sep 19 15:59:47 2022] GET /identity/v3/groups/92fde2624f7744c2b5e809954f75a8d9 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:47.678910 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d53015-7f8a-4b68-a269-806e4e6b5b33 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:47.766409 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabbacf0-3ee6-4da7-9c0c-62cd09591f38 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:47.805882 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d53015-7f8a-4b68-a269-806e4e6b5b33 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:47.819435 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e877b1-bcfd-4674-b139-c95ce2e50942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:47.820081 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e877b1-bcfd-4674-b139-c95ce2e50942 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:47.820358 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e877b1-bcfd-4674-b139-c95ce2e50942 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:47.820631 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e877b1-bcfd-4674-b139-c95ce2e50942 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:47.821599 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e877b1-bcfd-4674-b139-c95ce2e50942 None admin] RBAC: Adding query filter params (group.id=92fde2624f7744c2b5e809954f75a8d9, role.id=31b0628699ea4f98824f20e7568748cc, scope.project.id=ad7bddf539af4f8dabada63995dab07f) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:47.821823 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e877b1-bcfd-4674-b139-c95ce2e50942 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:47.823367 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e877b1-bcfd-4674-b139-c95ce2e50942 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:47.834316 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 184/357] 10.222.0.57 () {56 vars in 1376 bytes} [Mon Sep 19 15:59:47 2022] GET /identity/v3/role_assignments?group.id=92fde2624f7744c2b5e809954f75a8d9&scope.project.id=ad7bddf539af4f8dabada63995dab07f&role.id=31b0628699ea4f98824f20e7568748cc => generated 260 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:47.835175 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e877b1-bcfd-4674-b139-c95ce2e50942 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:47.854050 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9835f9-cce3-431e-a6c9-beb1158b2712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:47.855158 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9835f9-cce3-431e-a6c9-beb1158b2712 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:47.855661 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9835f9-cce3-431e-a6c9-beb1158b2712 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:47.856180 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9835f9-cce3-431e-a6c9-beb1158b2712 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:47.883918 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9835f9-cce3-431e-a6c9-beb1158b2712 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 15:59:47.891184 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-9e9835f9-cce3-431e-a6c9-beb1158b2712 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 15:59:47.891184 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 15:59:47.891184 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 15:59:47.891184 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 15:59:47.891184 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 15:59:47.891184 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 15:59:47.891184 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 15:59:47.891184 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 15:59:47.891184 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 15:59:47.891184 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 15:59:47.891184 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 15:59:47.891184 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 15:59:47.891184 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 15:59:47.891184 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 15:59:47.891184 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 15:59:47.891184 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 15:59:47.892120 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 15:59:47.892120 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 15:59:47.892120 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 15:59:47.892120 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 15:59:47.892120 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 15:59:47.892120 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 15:59:47.892120 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 15:59:47.892120 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 15:59:47.892120 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 15:59:47.892120 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 15:59:47.892120 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 15:59:47.892120 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 15:59:47.892120 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 15:59:47.892120 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 15:59:47.892120 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 15:59:47.892120 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 15:59:47.892928 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 15:59:47.892928 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 15:59:47.892928 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 15:59:47.892928 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 15:59:47.892928 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 15:59:47.892928 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 15:59:47.892928 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 15:59:47.892928 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 15:59:47.892928 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 15:59:47.892928 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 15:59:47.892928 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 15:59:47.892928 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 15:59:47.897874 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 174/358] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 15:59:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 15:59:47.900860 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9835f9-cce3-431e-a6c9-beb1158b2712 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:47.956843 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:47.964308 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:48.022917 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e877b1-bcfd-4674-b139-c95ce2e50942 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:48.063513 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9835f9-cce3-431e-a6c9-beb1158b2712 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:48.074925 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eac118-2093-4900-94e1-cfcb2cf34f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:48.075852 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eac118-2093-4900-94e1-cfcb2cf34f69 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:48.076402 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eac118-2093-4900-94e1-cfcb2cf34f69 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:48.076776 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eac118-2093-4900-94e1-cfcb2cf34f69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:48.103902 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26f3e06-da00-4c83-bc10-fec1b4ab7bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:48.104814 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f3e06-da00-4c83-bc10-fec1b4ab7bc1 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:48.105230 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f3e06-da00-4c83-bc10-fec1b4ab7bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:48.105685 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f3e06-da00-4c83-bc10-fec1b4ab7bc1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:48.106592 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26f3e06-da00-4c83-bc10-fec1b4ab7bc1 None admin] RBAC: Adding query filter params (group.id=92fde2624f7744c2b5e809954f75a8d9, role.id=a1606b79a2c840238de6860bc69b49b7, scope.project.id=ad7bddf539af4f8dabada63995dab07f) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:48.107064 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26f3e06-da00-4c83-bc10-fec1b4ab7bc1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:48.109388 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26f3e06-da00-4c83-bc10-fec1b4ab7bc1 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:48.115535 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 175/359] 10.222.0.57 () {56 vars in 1376 bytes} [Mon Sep 19 15:59:47 2022] GET /identity/v3/role_assignments?group.id=92fde2624f7744c2b5e809954f75a8d9&scope.project.id=ad7bddf539af4f8dabada63995dab07f&role.id=a1606b79a2c840238de6860bc69b49b7 => generated 260 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:48.116140 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26f3e06-da00-4c83-bc10-fec1b4ab7bc1 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:48.163927 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eac118-2093-4900-94e1-cfcb2cf34f69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:48.166013 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eac118-2093-4900-94e1-cfcb2cf34f69 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:48.246968 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:48.260186 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:48.261987 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26f3e06-da00-4c83-bc10-fec1b4ab7bc1 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:48.309324 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7d7487-66ea-4f47-89c8-1a17cc39c4cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:48.310235 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d7487-66ea-4f47-89c8-1a17cc39c4cd None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:48.311564 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d7487-66ea-4f47-89c8-1a17cc39c4cd None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:48.311983 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d7487-66ea-4f47-89c8-1a17cc39c4cd None admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:48.313548 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7d7487-66ea-4f47-89c8-1a17cc39c4cd None admin] RBAC: Adding query filter params (name=reader) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:48.314003 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7d7487-66ea-4f47-89c8-1a17cc39c4cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:48.315658 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7d7487-66ea-4f47-89c8-1a17cc39c4cd None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:48.316533 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 185/360] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 19 15:59:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 15:59:48.317480 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eac118-2093-4900-94e1-cfcb2cf34f69 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:48.326587 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 176/361] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Sep 19 15:59:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:48.328204 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c947a6-dfbf-484b-bc86-10d6949d2538 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:48.328648 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c947a6-dfbf-484b-bc86-10d6949d2538 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:48.329070 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c947a6-dfbf-484b-bc86-10d6949d2538 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:48.330828 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 177/362] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:59:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 15:59:48.331837 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c947a6-dfbf-484b-bc86-10d6949d2538 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:48.457565 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eac118-2093-4900-94e1-cfcb2cf34f69 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:48.480444 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c947a6-dfbf-484b-bc86-10d6949d2538 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:48.501547 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93085394-9d79-4fe5-83a8-ce2af38710ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:48.501963 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93085394-9d79-4fe5-83a8-ce2af38710ff None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:48.502247 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93085394-9d79-4fe5-83a8-ce2af38710ff None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:48.502735 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93085394-9d79-4fe5-83a8-ce2af38710ff None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:48.514194 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93085394-9d79-4fe5-83a8-ce2af38710ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 15:59:48.520801 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-93085394-9d79-4fe5-83a8-ce2af38710ff None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 15:59:48.520801 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 15:59:48.520801 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 15:59:48.520801 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 15:59:48.520801 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 15:59:48.520801 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 15:59:48.520801 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 15:59:48.520801 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 15:59:48.520801 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 15:59:48.520801 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 15:59:48.520801 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 15:59:48.520801 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 15:59:48.520801 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 15:59:48.520801 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 15:59:48.520801 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 15:59:48.520801 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 15:59:48.521791 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 15:59:48.521791 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 15:59:48.521791 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 15:59:48.521791 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 15:59:48.521791 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 15:59:48.521791 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 15:59:48.521791 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 15:59:48.521791 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 15:59:48.521791 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 15:59:48.521791 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 15:59:48.521791 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 15:59:48.521791 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 15:59:48.521791 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 15:59:48.521791 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 15:59:48.521791 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 15:59:48.521791 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 15:59:48.523162 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 15:59:48.523162 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 15:59:48.523162 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 15:59:48.523162 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 15:59:48.523162 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 15:59:48.523162 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 15:59:48.523162 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 15:59:48.523162 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 15:59:48.523162 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 15:59:48.523162 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 15:59:48.523162 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 15:59:48.523162 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 15:59:48.523162 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 186/363] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Sep 19 15:59:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 15:59:48.523162 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808fc03d-37ae-49c0-a5c5-9d04dcb7366a None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:48.523880 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808fc03d-37ae-49c0-a5c5-9d04dcb7366a None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:48.523880 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808fc03d-37ae-49c0-a5c5-9d04dcb7366a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:48.530947 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edd9e45-e3e1-4493-8eab-c77494dcfc84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:48.532214 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edd9e45-e3e1-4493-8eab-c77494dcfc84 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:48.533689 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edd9e45-e3e1-4493-8eab-c77494dcfc84 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:48.534171 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edd9e45-e3e1-4493-8eab-c77494dcfc84 None admin] PATH_INFO: `/v3/projects/93b0db50dd5e4d3195beded37dd4105f` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:48.535604 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edd9e45-e3e1-4493-8eab-c77494dcfc84 None admin] RBAC: Authorizing `identity:get_project(project_id=93b0db50dd5e4d3195beded37dd4105f)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:48.537242 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edd9e45-e3e1-4493-8eab-c77494dcfc84 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:48.539916 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 178/364] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 19 15:59:48 2022] GET /identity/v3/projects/93b0db50dd5e4d3195beded37dd4105f => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:48.541485 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edd9e45-e3e1-4493-8eab-c77494dcfc84 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:48.545836 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-808fc03d-37ae-49c0-a5c5-9d04dcb7366a None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:59:48.547488 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-808fc03d-37ae-49c0-a5c5-9d04dcb7366a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:59:48.565566 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:48.579194 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:48.636986 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 187/365] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:59:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 15:59:48.638304 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808fc03d-37ae-49c0-a5c5-9d04dcb7366a None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:48.675718 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edd9e45-e3e1-4493-8eab-c77494dcfc84 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:48.712323 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d65c9ec-abb5-4487-aa44-5e5947f5234a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:48.712883 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d65c9ec-abb5-4487-aa44-5e5947f5234a None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:48.713209 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d65c9ec-abb5-4487-aa44-5e5947f5234a None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:48.713483 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d65c9ec-abb5-4487-aa44-5e5947f5234a None admin] PATH_INFO: `/v3/users/1f336d28c20449c7b1ea7375dcdbd196` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:48.714741 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d65c9ec-abb5-4487-aa44-5e5947f5234a None admin] RBAC: Authorizing `identity:get_user(user_id=1f336d28c20449c7b1ea7375dcdbd196)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:48.717527 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d65c9ec-abb5-4487-aa44-5e5947f5234a None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:48.718767 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 179/366] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 19 15:59:48 2022] GET /identity/v3/users/1f336d28c20449c7b1ea7375dcdbd196 => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:48.719594 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d65c9ec-abb5-4487-aa44-5e5947f5234a None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:48.750189 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808fc03d-37ae-49c0-a5c5-9d04dcb7366a None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:48.788206 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2efeaa-7088-468b-aaf6-922c52a0bc1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:48.788926 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2efeaa-7088-468b-aaf6-922c52a0bc1e None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:48.789296 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2efeaa-7088-468b-aaf6-922c52a0bc1e None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:48.789667 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2efeaa-7088-468b-aaf6-922c52a0bc1e None admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:48.790486 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2efeaa-7088-468b-aaf6-922c52a0bc1e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:48.790897 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2efeaa-7088-468b-aaf6-922c52a0bc1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:48.792622 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2efeaa-7088-468b-aaf6-922c52a0bc1e None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:48.802338 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 188/367] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 19 15:59:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:48.803306 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2efeaa-7088-468b-aaf6-922c52a0bc1e None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:48.819983 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:48.834570 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:48.848712 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d65c9ec-abb5-4487-aa44-5e5947f5234a None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:48.885883 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4419a891-d0bd-461c-a171-9b896fb6683f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:48.886794 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4419a891-d0bd-461c-a171-9b896fb6683f None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:48.887218 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4419a891-d0bd-461c-a171-9b896fb6683f None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:48.887669 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4419a891-d0bd-461c-a171-9b896fb6683f None admin] PATH_INFO: `/v3/groups/92fde2624f7744c2b5e809954f75a8d9` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:48.889096 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4419a891-d0bd-461c-a171-9b896fb6683f None admin] RBAC: Authorizing `identity:get_group(group_id=92fde2624f7744c2b5e809954f75a8d9)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:48.890781 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4419a891-d0bd-461c-a171-9b896fb6683f None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:48.892856 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 180/368] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Sep 19 15:59:48 2022] GET /identity/v3/groups/92fde2624f7744c2b5e809954f75a8d9 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:48.893868 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4419a891-d0bd-461c-a171-9b896fb6683f None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:48.913626 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2efeaa-7088-468b-aaf6-922c52a0bc1e None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:48.950847 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ddb258-39b0-4cc0-80da-bf1445f6f011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:48.951871 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ddb258-39b0-4cc0-80da-bf1445f6f011 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:48.952243 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ddb258-39b0-4cc0-80da-bf1445f6f011 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:48.952613 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ddb258-39b0-4cc0-80da-bf1445f6f011 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:48.953475 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ddb258-39b0-4cc0-80da-bf1445f6f011 None admin] RBAC: Adding query filter params (role.id=625d911d4d934c1e80f0a57620077d49, user.id=1f336d28c20449c7b1ea7375dcdbd196, scope.system=all) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:48.953879 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ddb258-39b0-4cc0-80da-bf1445f6f011 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:48.955494 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ddb258-39b0-4cc0-80da-bf1445f6f011 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:48.974483 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 189/369] 10.222.0.57 () {56 vars in 1308 bytes} [Mon Sep 19 15:59:48 2022] GET /identity/v3/role_assignments?user.id=1f336d28c20449c7b1ea7375dcdbd196&scope.system=all&role.id=625d911d4d934c1e80f0a57620077d49 => generated 510 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:48.974483 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ddb258-39b0-4cc0-80da-bf1445f6f011 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:49.036659 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4419a891-d0bd-461c-a171-9b896fb6683f None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:49.072741 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:49.082711 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781df419-eab3-4ade-b60d-50504812d1d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:49.083346 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781df419-eab3-4ade-b60d-50504812d1d6 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:49.083629 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781df419-eab3-4ade-b60d-50504812d1d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:49.083908 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781df419-eab3-4ade-b60d-50504812d1d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:49.112667 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ddb258-39b0-4cc0-80da-bf1445f6f011 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:49.117167 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:49.152070 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b1d73d-86c1-4586-8fcb-8778d584f101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:49.153144 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b1d73d-86c1-4586-8fcb-8778d584f101 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:49.153641 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b1d73d-86c1-4586-8fcb-8778d584f101 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:49.154423 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b1d73d-86c1-4586-8fcb-8778d584f101 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:49.155848 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b1d73d-86c1-4586-8fcb-8778d584f101 None admin] RBAC: Adding query filter params (group.id=92fde2624f7744c2b5e809954f75a8d9, role.id=31b0628699ea4f98824f20e7568748cc, scope.project.id=93b0db50dd5e4d3195beded37dd4105f) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:49.156159 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b1d73d-86c1-4586-8fcb-8778d584f101 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:49.157639 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b1d73d-86c1-4586-8fcb-8778d584f101 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:49.163612 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781df419-eab3-4ade-b60d-50504812d1d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:49.164287 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 190/370] 10.222.0.57 () {56 vars in 1376 bytes} [Mon Sep 19 15:59:48 2022] GET /identity/v3/role_assignments?group.id=92fde2624f7744c2b5e809954f75a8d9&scope.project.id=93b0db50dd5e4d3195beded37dd4105f&role.id=31b0628699ea4f98824f20e7568748cc => generated 260 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:49.165083 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781df419-eab3-4ade-b60d-50504812d1d6 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:49.165205 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b1d73d-86c1-4586-8fcb-8778d584f101 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:49.282325 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 181/371] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 19 15:59:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 15:59:49.288695 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781df419-eab3-4ade-b60d-50504812d1d6 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:49.391977 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b1d73d-86c1-4586-8fcb-8778d584f101 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:49.438903 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:49.468483 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233f34fc-efaf-46c5-80e0-47feeac14ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:49.469134 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f34fc-efaf-46c5-80e0-47feeac14ad3 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:49.469407 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f34fc-efaf-46c5-80e0-47feeac14ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:49.469674 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f34fc-efaf-46c5-80e0-47feeac14ad3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:49.472439 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:49.491748 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233f34fc-efaf-46c5-80e0-47feeac14ad3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 15:59:49.498699 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-233f34fc-efaf-46c5-80e0-47feeac14ad3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 15:59:49.498699 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 15:59:49.498699 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 15:59:49.498699 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 15:59:49.498699 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 15:59:49.498699 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 15:59:49.498699 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 15:59:49.498699 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 15:59:49.498699 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 15:59:49.498699 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 15:59:49.498699 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 15:59:49.498699 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 15:59:49.498699 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 15:59:49.498699 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 15:59:49.498699 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 15:59:49.498699 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 15:59:49.500232 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 15:59:49.500232 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 15:59:49.500232 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 15:59:49.500232 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 15:59:49.500232 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 15:59:49.500232 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 15:59:49.500232 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 15:59:49.500232 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 15:59:49.500232 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 15:59:49.500232 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 15:59:49.500232 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 15:59:49.500232 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 15:59:49.500232 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 15:59:49.500232 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 15:59:49.500232 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 15:59:49.500232 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 15:59:49.501530 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 15:59:49.501530 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 15:59:49.501530 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 15:59:49.501530 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 15:59:49.501530 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 15:59:49.501530 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 15:59:49.501530 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 15:59:49.501530 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 15:59:49.501530 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 15:59:49.501530 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 15:59:49.501530 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 15:59:49.501530 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 15:59:49.501530 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 191/372] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 19 15:59:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 15:59:49.506634 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233f34fc-efaf-46c5-80e0-47feeac14ad3 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:49.512003 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781df419-eab3-4ade-b60d-50504812d1d6 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:49.553537 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc856784-1529-48a7-b344-6b04b57b0a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:49.554096 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc856784-1529-48a7-b344-6b04b57b0a59 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:49.556253 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc856784-1529-48a7-b344-6b04b57b0a59 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:49.556253 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc856784-1529-48a7-b344-6b04b57b0a59 None admin] PATH_INFO: `/v3/projects/93b0db50dd5e4d3195beded37dd4105f` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:49.559126 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc856784-1529-48a7-b344-6b04b57b0a59 None admin] RBAC: Authorizing `identity:get_project(project_id=93b0db50dd5e4d3195beded37dd4105f)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:49.560606 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc856784-1529-48a7-b344-6b04b57b0a59 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:49.562607 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 182/373] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 19 15:59:49 2022] GET /identity/v3/projects/93b0db50dd5e4d3195beded37dd4105f => generated 302 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 15:59:49.578240 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc856784-1529-48a7-b344-6b04b57b0a59 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:49.690897 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233f34fc-efaf-46c5-80e0-47feeac14ad3 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:49.692272 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc856784-1529-48a7-b344-6b04b57b0a59 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:49.721423 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:49.738815 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d581b75c-d05d-4ccd-a48d-5ac2701c2e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:49.739451 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d581b75c-d05d-4ccd-a48d-5ac2701c2e50 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:49.739722 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d581b75c-d05d-4ccd-a48d-5ac2701c2e50 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:49.739987 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d581b75c-d05d-4ccd-a48d-5ac2701c2e50 None admin] PATH_INFO: `/v3/groups/92fde2624f7744c2b5e809954f75a8d9` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:49.741826 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d581b75c-d05d-4ccd-a48d-5ac2701c2e50 None admin] RBAC: Authorizing `identity:get_group(group_id=92fde2624f7744c2b5e809954f75a8d9)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:49.743427 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d581b75c-d05d-4ccd-a48d-5ac2701c2e50 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:49.745211 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 183/374] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Sep 19 15:59:49 2022] GET /identity/v3/groups/92fde2624f7744c2b5e809954f75a8d9 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 15:59:49.751538 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d581b75c-d05d-4ccd-a48d-5ac2701c2e50 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:49.766550 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dec9f1-633b-4f7f-ba6f-7c8c0afebdf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:49.767314 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dec9f1-633b-4f7f-ba6f-7c8c0afebdf7 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:49.767596 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dec9f1-633b-4f7f-ba6f-7c8c0afebdf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:49.767868 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dec9f1-633b-4f7f-ba6f-7c8c0afebdf7 None admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:49.768555 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dec9f1-633b-4f7f-ba6f-7c8c0afebdf7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:49.768856 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dec9f1-633b-4f7f-ba6f-7c8c0afebdf7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:49.770272 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dec9f1-633b-4f7f-ba6f-7c8c0afebdf7 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:49.787015 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 192/375] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 15:59:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:49.794506 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dec9f1-633b-4f7f-ba6f-7c8c0afebdf7 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:49.883879 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d581b75c-d05d-4ccd-a48d-5ac2701c2e50 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:49.930723 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:49.972813 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47dcda6-23bd-430b-b258-1d641124f39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:49.973856 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47dcda6-23bd-430b-b258-1d641124f39a None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:49.974347 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47dcda6-23bd-430b-b258-1d641124f39a None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:49.974856 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47dcda6-23bd-430b-b258-1d641124f39a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:49.977446 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47dcda6-23bd-430b-b258-1d641124f39a None admin] RBAC: Adding query filter params (group.id=92fde2624f7744c2b5e809954f75a8d9, role.id=a1606b79a2c840238de6860bc69b49b7, scope.project.id=93b0db50dd5e4d3195beded37dd4105f) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:49.977995 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47dcda6-23bd-430b-b258-1d641124f39a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:49.979676 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47dcda6-23bd-430b-b258-1d641124f39a None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:49.994534 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 184/376] 10.222.0.57 () {56 vars in 1376 bytes} [Mon Sep 19 15:59:49 2022] GET /identity/v3/role_assignments?group.id=92fde2624f7744c2b5e809954f75a8d9&scope.project.id=93b0db50dd5e4d3195beded37dd4105f&role.id=a1606b79a2c840238de6860bc69b49b7 => generated 260 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:50.013586 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dec9f1-633b-4f7f-ba6f-7c8c0afebdf7 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:50.079012 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfd63c8-8784-4cf3-a887-390d5712fe16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:50.080020 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfd63c8-8784-4cf3-a887-390d5712fe16 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:50.080497 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfd63c8-8784-4cf3-a887-390d5712fe16 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:50.080993 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfd63c8-8784-4cf3-a887-390d5712fe16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:50.181529 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfd63c8-8784-4cf3-a887-390d5712fe16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:50.182986 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfd63c8-8784-4cf3-a887-390d5712fe16 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:50.261702 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:50.331976 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 193/377] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 19 15:59:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 15:59:50.338063 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47dcda6-23bd-430b-b258-1d641124f39a None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:50.447721 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:50.489401 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47dcda6-23bd-430b-b258-1d641124f39a None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:50.532723 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5855a2c-b957-46b0-aa0a-ab849f18516c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:50.533365 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5855a2c-b957-46b0-aa0a-ab849f18516c None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:50.533708 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5855a2c-b957-46b0-aa0a-ab849f18516c None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:50.533989 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5855a2c-b957-46b0-aa0a-ab849f18516c None admin] PATH_INFO: `/v3/projects/5a804037a8734a35ad3ef5ec7e34d4c6` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:50.535170 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5855a2c-b957-46b0-aa0a-ab849f18516c None admin] RBAC: Authorizing `identity:get_project(project_id=5a804037a8734a35ad3ef5ec7e34d4c6)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:50.536598 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5855a2c-b957-46b0-aa0a-ab849f18516c None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:50.538591 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 185/378] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 19 15:59:50 2022] GET /identity/v3/projects/5a804037a8734a35ad3ef5ec7e34d4c6 => generated 344 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:50.544431 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfd63c8-8784-4cf3-a887-390d5712fe16 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:50.704050 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfd63c8-8784-4cf3-a887-390d5712fe16 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:50.749608 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066f9700-9afa-48c9-a1f9-dbe5f27d9954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:50.750690 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f9700-9afa-48c9-a1f9-dbe5f27d9954 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:50.751177 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f9700-9afa-48c9-a1f9-dbe5f27d9954 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:50.751671 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f9700-9afa-48c9-a1f9-dbe5f27d9954 None admin] PATH_INFO: `/v3/groups/716a6148123142eca9ea2581da7f2458` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:50.759045 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066f9700-9afa-48c9-a1f9-dbe5f27d9954 None admin] RBAC: Authorizing `identity:get_group(group_id=716a6148123142eca9ea2581da7f2458)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:50.760942 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066f9700-9afa-48c9-a1f9-dbe5f27d9954 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:50.763040 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 194/379] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Sep 19 15:59:50 2022] GET /identity/v3/groups/716a6148123142eca9ea2581da7f2458 => generated 229 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:50.771894 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5855a2c-b957-46b0-aa0a-ab849f18516c None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:50.920270 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5855a2c-b957-46b0-aa0a-ab849f18516c None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:50.956980 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:50.966826 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a6de0e-ab26-4898-bc99-aceb87824533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:50.967445 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6de0e-ab26-4898-bc99-aceb87824533 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:50.967732 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6de0e-ab26-4898-bc99-aceb87824533 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:50.968007 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6de0e-ab26-4898-bc99-aceb87824533 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:50.968739 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a6de0e-ab26-4898-bc99-aceb87824533 None admin] RBAC: Adding query filter params (group.id=716a6148123142eca9ea2581da7f2458, role.id=63d2d197bfd6497583e94bfb065ce29b, scope.project.id=5a804037a8734a35ad3ef5ec7e34d4c6) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:50.969046 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a6de0e-ab26-4898-bc99-aceb87824533 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:50.970500 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a6de0e-ab26-4898-bc99-aceb87824533 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:50.981314 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 186/380] 10.222.0.57 () {56 vars in 1376 bytes} [Mon Sep 19 15:59:50 2022] GET /identity/v3/role_assignments?group.id=716a6148123142eca9ea2581da7f2458&scope.project.id=5a804037a8734a35ad3ef5ec7e34d4c6&role.id=63d2d197bfd6497583e94bfb065ce29b => generated 260 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:51.118534 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a3f3a6-9620-439e-95db-3a26e9d09959 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:51.118858 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a3f3a6-9620-439e-95db-3a26e9d09959 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:51.119144 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a3f3a6-9620-439e-95db-3a26e9d09959 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:51.120755 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 195/381] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:59:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 15:59:51.127722 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3e0dfc-5d9a-4f10-8f8f-3ce3beff85a0 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:51.128084 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3e0dfc-5d9a-4f10-8f8f-3ce3beff85a0 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:51.128359 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3e0dfc-5d9a-4f10-8f8f-3ce3beff85a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:51.155599 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-4f3e0dfc-5d9a-4f10-8f8f-3ce3beff85a0 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:59:51.157213 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-4f3e0dfc-5d9a-4f10-8f8f-3ce3beff85a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:59:51.256862 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 187/382] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:59:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 15:59:51.569599 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddbf7b2-b4fa-4ab7-aea1-b905fa9d0d71 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:51.570083 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddbf7b2-b4fa-4ab7-aea1-b905fa9d0d71 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:51.570083 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddbf7b2-b4fa-4ab7-aea1-b905fa9d0d71 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:51.571731 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 196/383] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:59:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 15:59:51.577610 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1dc8e3-3e7f-4388-bbad-bdec7cc42603 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:51.578534 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1dc8e3-3e7f-4388-bbad-bdec7cc42603 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:51.579071 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1dc8e3-3e7f-4388-bbad-bdec7cc42603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:51.607446 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-3f1dc8e3-3e7f-4388-bbad-bdec7cc42603 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:59:51.608989 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-3f1dc8e3-3e7f-4388-bbad-bdec7cc42603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:59:51.632714 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddbf7b2-b4fa-4ab7-aea1-b905fa9d0d71 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:51.677580 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:51.693553 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 188/384] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:59:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 15:59:51.811185 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddbf7b2-b4fa-4ab7-aea1-b905fa9d0d71 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:51.861033 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc91daf9-6134-4571-9cfe-77a24d438d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:51.861598 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91daf9-6134-4571-9cfe-77a24d438d23 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:51.861874 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91daf9-6134-4571-9cfe-77a24d438d23 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:51.862144 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91daf9-6134-4571-9cfe-77a24d438d23 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:51.883594 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc91daf9-6134-4571-9cfe-77a24d438d23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 15:59:51.888464 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-bc91daf9-6134-4571-9cfe-77a24d438d23 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 15:59:51.888464 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 15:59:51.888464 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 15:59:51.888464 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 15:59:51.888464 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 15:59:51.888464 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 15:59:51.888464 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 15:59:51.888464 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 15:59:51.888464 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 15:59:51.888464 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 15:59:51.888464 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 15:59:51.888464 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 15:59:51.888464 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 15:59:51.888464 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 15:59:51.888464 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 15:59:51.888464 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 15:59:51.890227 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 15:59:51.890227 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 15:59:51.890227 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 15:59:51.890227 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 15:59:51.890227 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 15:59:51.890227 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 15:59:51.890227 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 15:59:51.890227 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 15:59:51.890227 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 15:59:51.890227 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 15:59:51.890227 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 15:59:51.890227 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 15:59:51.890227 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 15:59:51.890227 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 15:59:51.890227 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 15:59:51.890227 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 15:59:51.891040 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 15:59:51.891040 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 15:59:51.891040 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 15:59:51.891040 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 15:59:51.891040 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 15:59:51.891040 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 15:59:51.891040 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 15:59:51.891040 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 15:59:51.891040 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 15:59:51.891040 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 15:59:51.891040 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 15:59:51.891040 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 15:59:51.891641 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 197/385] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 15:59:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 15:59:51.899489 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1dc8e3-3e7f-4388-bbad-bdec7cc42603 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:51.971958 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:52.035746 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1dc8e3-3e7f-4388-bbad-bdec7cc42603 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:52.089916 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3621f89f-6a65-4be2-ac56-c5d77cf78731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:52.090555 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3621f89f-6a65-4be2-ac56-c5d77cf78731 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:52.090834 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3621f89f-6a65-4be2-ac56-c5d77cf78731 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:52.091104 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3621f89f-6a65-4be2-ac56-c5d77cf78731 None admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:52.091799 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3621f89f-6a65-4be2-ac56-c5d77cf78731 None admin] RBAC: Adding query filter params (name=member) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:52.092102 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3621f89f-6a65-4be2-ac56-c5d77cf78731 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:52.093455 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3621f89f-6a65-4be2-ac56-c5d77cf78731 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:52.102459 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 189/386] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Sep 19 15:59:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:52.111536 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc91daf9-6134-4571-9cfe-77a24d438d23 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:52.181923 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3621f89f-6a65-4be2-ac56-c5d77cf78731 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:52.245658 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:52.278508 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc91daf9-6134-4571-9cfe-77a24d438d23 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:52.318068 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3621f89f-6a65-4be2-ac56-c5d77cf78731 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:52.327549 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fb6d63-b1a3-4805-880d-b42da707ee41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:52.328554 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb6d63-b1a3-4805-880d-b42da707ee41 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:52.329591 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb6d63-b1a3-4805-880d-b42da707ee41 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:52.330069 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb6d63-b1a3-4805-880d-b42da707ee41 None admin] PATH_INFO: `/v3/groups/92fde2624f7744c2b5e809954f75a8d9` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:52.332125 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fb6d63-b1a3-4805-880d-b42da707ee41 None admin] RBAC: Authorizing `identity:get_group(group_id=92fde2624f7744c2b5e809954f75a8d9)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:52.333305 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:52.334991 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fb6d63-b1a3-4805-880d-b42da707ee41 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:52.337750 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 198/387] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Sep 19 15:59:52 2022] GET /identity/v3/groups/92fde2624f7744c2b5e809954f75a8d9 => generated 226 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:52.339461 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ff05f-be54-4919-9243-9e3d8ba2f6c6 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:52.339461 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ff05f-be54-4919-9243-9e3d8ba2f6c6 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:52.339896 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ff05f-be54-4919-9243-9e3d8ba2f6c6 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:52.342710 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 199/388] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:59:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 15:59:52.343462 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ff05f-be54-4919-9243-9e3d8ba2f6c6 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:52.378806 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835484a0-e24f-4e4a-95ca-49c6ecfb4bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:52.380155 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835484a0-e24f-4e4a-95ca-49c6ecfb4bfe None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:52.381134 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835484a0-e24f-4e4a-95ca-49c6ecfb4bfe None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:52.381634 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835484a0-e24f-4e4a-95ca-49c6ecfb4bfe None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:52.398662 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835484a0-e24f-4e4a-95ca-49c6ecfb4bfe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 15:59:52.405754 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-835484a0-e24f-4e4a-95ca-49c6ecfb4bfe None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 15:59:52.405754 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 15:59:52.405754 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 15:59:52.405754 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 15:59:52.405754 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 15:59:52.405754 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 15:59:52.405754 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 15:59:52.405754 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 15:59:52.405754 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 15:59:52.405754 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 15:59:52.405754 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 15:59:52.405754 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 15:59:52.405754 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 15:59:52.405754 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 15:59:52.405754 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 15:59:52.405754 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 15:59:52.406691 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 15:59:52.406691 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 15:59:52.406691 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 15:59:52.406691 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 15:59:52.406691 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 15:59:52.406691 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 15:59:52.406691 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 15:59:52.406691 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 15:59:52.406691 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 15:59:52.406691 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 15:59:52.406691 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 15:59:52.406691 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 15:59:52.406691 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 15:59:52.406691 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 15:59:52.406691 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 15:59:52.406691 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 15:59:52.407469 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 15:59:52.407469 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 15:59:52.407469 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 15:59:52.407469 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 15:59:52.407469 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 15:59:52.407469 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 15:59:52.407469 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 15:59:52.407469 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 15:59:52.407469 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 15:59:52.407469 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 15:59:52.407469 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 15:59:52.407469 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 15:59:52.407469 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 190/389] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Sep 19 15:59:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 15:59:52.408711 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7678826-ac74-4ceb-82a7-5560c1d6425f None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:52.408711 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7678826-ac74-4ceb-82a7-5560c1d6425f None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:52.408870 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7678826-ac74-4ceb-82a7-5560c1d6425f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:52.428688 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-b7678826-ac74-4ceb-82a7-5560c1d6425f None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:59:52.429991 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-b7678826-ac74-4ceb-82a7-5560c1d6425f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:59:52.497010 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ff05f-be54-4919-9243-9e3d8ba2f6c6 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:52.507912 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 191/390] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:59:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 15:59:52.508764 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7678826-ac74-4ceb-82a7-5560c1d6425f None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:52.561159 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea773c3-076a-4c67-9561-5031cf1fdd6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:52.562279 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea773c3-076a-4c67-9561-5031cf1fdd6e None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:52.562779 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea773c3-076a-4c67-9561-5031cf1fdd6e None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:52.563194 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea773c3-076a-4c67-9561-5031cf1fdd6e None admin] PATH_INFO: `/v3/projects/ad7bddf539af4f8dabada63995dab07f` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:52.564591 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea773c3-076a-4c67-9561-5031cf1fdd6e None admin] RBAC: Authorizing `identity:get_project(project_id=ad7bddf539af4f8dabada63995dab07f)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:52.570405 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea773c3-076a-4c67-9561-5031cf1fdd6e None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:52.573351 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 200/391] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 19 15:59:52 2022] GET /identity/v3/projects/ad7bddf539af4f8dabada63995dab07f => generated 298 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:52.579438 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea773c3-076a-4c67-9561-5031cf1fdd6e None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:52.604572 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:52.633646 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7678826-ac74-4ceb-82a7-5560c1d6425f None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:52.635992 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:52.681776 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b1270c-8556-4fc0-9a7d-2e5b003730cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:52.685519 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b1270c-8556-4fc0-9a7d-2e5b003730cf None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:52.685945 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b1270c-8556-4fc0-9a7d-2e5b003730cf None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:52.686423 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b1270c-8556-4fc0-9a7d-2e5b003730cf None admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:52.687274 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b1270c-8556-4fc0-9a7d-2e5b003730cf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:52.687739 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b1270c-8556-4fc0-9a7d-2e5b003730cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:52.689238 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b1270c-8556-4fc0-9a7d-2e5b003730cf None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:52.711369 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 192/392] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 19 15:59:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:52.711369 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3884f478-2a5e-406c-a33c-f68fcd89be39 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:52.711369 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3884f478-2a5e-406c-a33c-f68fcd89be39 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:52.711369 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3884f478-2a5e-406c-a33c-f68fcd89be39 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:52.713617 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 193/393] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:59:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 15:59:52.714207 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3884f478-2a5e-406c-a33c-f68fcd89be39 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:52.789973 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea773c3-076a-4c67-9561-5031cf1fdd6e None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:52.853875 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f6e8fa-6229-48e1-b19b-94718f44a0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:52.854874 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f6e8fa-6229-48e1-b19b-94718f44a0ea None admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:52.855154 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f6e8fa-6229-48e1-b19b-94718f44a0ea None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:52.855895 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f6e8fa-6229-48e1-b19b-94718f44a0ea None admin] PATH_INFO: `/v3/projects/ad7bddf539af4f8dabada63995dab07f/groups/92fde2624f7744c2b5e809954f75a8d9/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:52.859148 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f6e8fa-6229-48e1-b19b-94718f44a0ea None admin] RBAC: Authorizing `identity:create_grant(project_id=ad7bddf539af4f8dabada63995dab07f, group_id=92fde2624f7744c2b5e809954f75a8d9, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:52.861240 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f6e8fa-6229-48e1-b19b-94718f44a0ea None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:52.874356 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3884f478-2a5e-406c-a33c-f68fcd89be39 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:52.879314 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 201/394] 10.222.0.57 () {58 vars in 1442 bytes} [Mon Sep 19 15:59:52 2022] PUT /identity/v3/projects/ad7bddf539af4f8dabada63995dab07f/groups/92fde2624f7744c2b5e809954f75a8d9/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 15:59:52.880686 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc8d7c7-a7d7-41e1-9d87-b1f23ea8b91a None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:52.880984 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc8d7c7-a7d7-41e1-9d87-b1f23ea8b91a None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:52.881286 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc8d7c7-a7d7-41e1-9d87-b1f23ea8b91a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:52.905836 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-bbc8d7c7-a7d7-41e1-9d87-b1f23ea8b91a None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:59:52.907860 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-bbc8d7c7-a7d7-41e1-9d87-b1f23ea8b91a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:59:52.934092 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bc8169-437f-4b2f-acae-f2a7a66b5bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:52.934823 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc8169-437f-4b2f-acae-f2a7a66b5bdd None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:52.935153 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc8169-437f-4b2f-acae-f2a7a66b5bdd None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:52.935449 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc8169-437f-4b2f-acae-f2a7a66b5bdd None admin] PATH_INFO: `/v3/groups/92fde2624f7744c2b5e809954f75a8d9` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:52.938219 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bc8169-437f-4b2f-acae-f2a7a66b5bdd None admin] RBAC: Authorizing `identity:get_group(group_id=92fde2624f7744c2b5e809954f75a8d9)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:52.939864 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bc8169-437f-4b2f-acae-f2a7a66b5bdd None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:52.941688 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 194/395] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Sep 19 15:59:52 2022] GET /identity/v3/groups/92fde2624f7744c2b5e809954f75a8d9 => generated 226 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:52.942800 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bc8169-437f-4b2f-acae-f2a7a66b5bdd None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:52.992049 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:53.041043 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 202/396] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:59:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 15:59:53.046863 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc8d7c7-a7d7-41e1-9d87-b1f23ea8b91a None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:53.059390 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:53.086325 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bc8169-437f-4b2f-acae-f2a7a66b5bdd None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:53.129199 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8421ea-a100-4873-8599-d84e875b8175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:53.129836 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8421ea-a100-4873-8599-d84e875b8175 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:53.130113 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8421ea-a100-4873-8599-d84e875b8175 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:53.130408 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8421ea-a100-4873-8599-d84e875b8175 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:53.144704 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8421ea-a100-4873-8599-d84e875b8175 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 15:59:53.150699 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-aa8421ea-a100-4873-8599-d84e875b8175 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 15:59:53.150699 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 15:59:53.150699 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 15:59:53.150699 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 15:59:53.150699 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 15:59:53.150699 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 15:59:53.150699 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 15:59:53.150699 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 15:59:53.150699 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 15:59:53.150699 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 15:59:53.150699 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 15:59:53.150699 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 15:59:53.150699 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 15:59:53.150699 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 15:59:53.150699 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 15:59:53.150699 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 15:59:53.152671 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 15:59:53.152671 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 15:59:53.152671 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 15:59:53.152671 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 15:59:53.152671 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 15:59:53.152671 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 15:59:53.152671 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 15:59:53.152671 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 15:59:53.152671 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 15:59:53.152671 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 15:59:53.152671 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 15:59:53.152671 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 15:59:53.152671 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 15:59:53.152671 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 15:59:53.152671 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 15:59:53.152671 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 15:59:53.153986 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 15:59:53.153986 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 15:59:53.153986 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 15:59:53.153986 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 15:59:53.153986 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 15:59:53.153986 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 15:59:53.153986 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 15:59:53.153986 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 15:59:53.153986 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 15:59:53.153986 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 15:59:53.153986 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 15:59:53.153986 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 15:59:53.153986 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 195/397] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 15:59:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 15:59:53.159000 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8421ea-a100-4873-8599-d84e875b8175 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:53.195089 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc8d7c7-a7d7-41e1-9d87-b1f23ea8b91a None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:53.235004 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cd3f9f-f054-453f-aec4-86fe8ee6b2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:53.235658 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd3f9f-f054-453f-aec4-86fe8ee6b2d8 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:53.235940 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd3f9f-f054-453f-aec4-86fe8ee6b2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:53.236214 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd3f9f-f054-453f-aec4-86fe8ee6b2d8 None admin] PATH_INFO: `/v3/projects/ad7bddf539af4f8dabada63995dab07f` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:53.237498 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cd3f9f-f054-453f-aec4-86fe8ee6b2d8 None admin] RBAC: Authorizing `identity:get_project(project_id=ad7bddf539af4f8dabada63995dab07f)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:53.239132 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cd3f9f-f054-453f-aec4-86fe8ee6b2d8 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:53.240921 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 203/398] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 19 15:59:53 2022] GET /identity/v3/projects/ad7bddf539af4f8dabada63995dab07f => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:53.246883 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cd3f9f-f054-453f-aec4-86fe8ee6b2d8 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:53.266510 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:53.280020 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8421ea-a100-4873-8599-d84e875b8175 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:53.319202 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2058728-f409-4202-9cfb-9653fefe8575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:53.319822 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2058728-f409-4202-9cfb-9653fefe8575 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:53.320098 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2058728-f409-4202-9cfb-9653fefe8575 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:53.320368 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2058728-f409-4202-9cfb-9653fefe8575 None admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:53.321032 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2058728-f409-4202-9cfb-9653fefe8575 None admin] RBAC: Adding query filter params (name=member) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:53.322133 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2058728-f409-4202-9cfb-9653fefe8575 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:53.323011 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2058728-f409-4202-9cfb-9653fefe8575 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:53.333163 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 196/399] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Sep 19 15:59:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:53.338351 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2058728-f409-4202-9cfb-9653fefe8575 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:53.383311 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:53.434404 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cd3f9f-f054-453f-aec4-86fe8ee6b2d8 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:53.454019 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2058728-f409-4202-9cfb-9653fefe8575 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:53.480381 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca144b5b-b4bd-4023-9c9c-c8cb6466a9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:53.481063 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca144b5b-b4bd-4023-9c9c-c8cb6466a9bd None admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:53.481342 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca144b5b-b4bd-4023-9c9c-c8cb6466a9bd None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:53.481607 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca144b5b-b4bd-4023-9c9c-c8cb6466a9bd None admin] PATH_INFO: `/v3/projects/ad7bddf539af4f8dabada63995dab07f/groups/92fde2624f7744c2b5e809954f75a8d9/roles/a1606b79a2c840238de6860bc69b49b7` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:53.484084 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca144b5b-b4bd-4023-9c9c-c8cb6466a9bd None admin] RBAC: Authorizing `identity:create_grant(project_id=ad7bddf539af4f8dabada63995dab07f, group_id=92fde2624f7744c2b5e809954f75a8d9, role_id=a1606b79a2c840238de6860bc69b49b7)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:53.485695 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca144b5b-b4bd-4023-9c9c-c8cb6466a9bd None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:53.495894 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b924503f-4ec5-4091-9421-b812044ad502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:53.496553 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b924503f-4ec5-4091-9421-b812044ad502 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:53.496968 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b924503f-4ec5-4091-9421-b812044ad502 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:53.497267 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b924503f-4ec5-4091-9421-b812044ad502 None admin] PATH_INFO: `/v3/groups/92fde2624f7744c2b5e809954f75a8d9` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:53.498589 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b924503f-4ec5-4091-9421-b812044ad502 None admin] RBAC: Authorizing `identity:get_group(group_id=92fde2624f7744c2b5e809954f75a8d9)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:53.500157 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b924503f-4ec5-4091-9421-b812044ad502 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:53.502804 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 197/400] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Sep 19 15:59:53 2022] GET /identity/v3/groups/92fde2624f7744c2b5e809954f75a8d9 => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:53.508254 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b924503f-4ec5-4091-9421-b812044ad502 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:53.512509 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 204/401] 10.222.0.57 () {58 vars in 1442 bytes} [Mon Sep 19 15:59:53 2022] PUT /identity/v3/projects/ad7bddf539af4f8dabada63995dab07f/groups/92fde2624f7744c2b5e809954f75a8d9/roles/a1606b79a2c840238de6860bc69b49b7 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 15:59:53.541629 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca144b5b-b4bd-4023-9c9c-c8cb6466a9bd None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:53.549469 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:53.704374 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca144b5b-b4bd-4023-9c9c-c8cb6466a9bd None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:53.750492 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99c6329-dfb2-4a2e-acfe-17d757881e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:53.751169 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c6329-dfb2-4a2e-acfe-17d757881e60 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:53.751453 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c6329-dfb2-4a2e-acfe-17d757881e60 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:53.751730 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c6329-dfb2-4a2e-acfe-17d757881e60 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:53.756950 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:53.780979 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b924503f-4ec5-4091-9421-b812044ad502 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:53.789666 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99c6329-dfb2-4a2e-acfe-17d757881e60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 15:59:53.800593 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-a99c6329-dfb2-4a2e-acfe-17d757881e60 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 15:59:53.800593 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 15:59:53.800593 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 15:59:53.800593 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 15:59:53.800593 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 15:59:53.800593 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 15:59:53.800593 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 15:59:53.800593 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 15:59:53.800593 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 15:59:53.800593 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 15:59:53.800593 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 15:59:53.800593 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 15:59:53.800593 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 15:59:53.800593 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 15:59:53.800593 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 15:59:53.800593 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 15:59:53.802781 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 15:59:53.802781 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 15:59:53.802781 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 15:59:53.802781 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 15:59:53.802781 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 15:59:53.802781 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 15:59:53.802781 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 15:59:53.802781 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 15:59:53.802781 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 15:59:53.802781 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 15:59:53.802781 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 15:59:53.802781 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 15:59:53.802781 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 15:59:53.802781 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 15:59:53.802781 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 15:59:53.802781 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 15:59:53.804741 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 15:59:53.804741 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 15:59:53.804741 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 15:59:53.804741 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 15:59:53.804741 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 15:59:53.804741 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 15:59:53.804741 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 15:59:53.804741 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 15:59:53.804741 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 15:59:53.804741 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 15:59:53.804741 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 15:59:53.804741 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 15:59:53.809275 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 205/402] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Sep 19 15:59:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 15:59:53.810154 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99c6329-dfb2-4a2e-acfe-17d757881e60 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:53.836001 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adb832f-e7f7-4e9c-812b-67c72a9db335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:53.836734 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb832f-e7f7-4e9c-812b-67c72a9db335 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:53.837019 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb832f-e7f7-4e9c-812b-67c72a9db335 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:53.837346 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb832f-e7f7-4e9c-812b-67c72a9db335 None admin] PATH_INFO: `/v3/projects/93b0db50dd5e4d3195beded37dd4105f` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:53.839349 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adb832f-e7f7-4e9c-812b-67c72a9db335 None admin] RBAC: Authorizing `identity:get_project(project_id=93b0db50dd5e4d3195beded37dd4105f)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:53.840865 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adb832f-e7f7-4e9c-812b-67c72a9db335 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:53.844545 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 198/403] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 19 15:59:53 2022] GET /identity/v3/projects/93b0db50dd5e4d3195beded37dd4105f => generated 302 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:53.849223 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adb832f-e7f7-4e9c-812b-67c72a9db335 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:53.957523 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99c6329-dfb2-4a2e-acfe-17d757881e60 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:53.974405 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:53.990619 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adb832f-e7f7-4e9c-812b-67c72a9db335 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:54.015124 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53da7c3-db59-49e0-805b-910ed80b2450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:54.016543 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53da7c3-db59-49e0-805b-910ed80b2450 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:54.017629 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53da7c3-db59-49e0-805b-910ed80b2450 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:54.018142 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53da7c3-db59-49e0-805b-910ed80b2450 None admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:54.019801 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53da7c3-db59-49e0-805b-910ed80b2450 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:54.020704 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53da7c3-db59-49e0-805b-910ed80b2450 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:54.025179 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53da7c3-db59-49e0-805b-910ed80b2450 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:54.030615 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374dd048-b4db-4dc7-8cf1-9dadf6512a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:54.031544 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374dd048-b4db-4dc7-8cf1-9dadf6512a38 None admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:54.032061 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374dd048-b4db-4dc7-8cf1-9dadf6512a38 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:54.032551 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374dd048-b4db-4dc7-8cf1-9dadf6512a38 None admin] PATH_INFO: `/v3/projects/93b0db50dd5e4d3195beded37dd4105f/groups/92fde2624f7744c2b5e809954f75a8d9/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:54.034758 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374dd048-b4db-4dc7-8cf1-9dadf6512a38 None admin] RBAC: Authorizing `identity:create_grant(project_id=93b0db50dd5e4d3195beded37dd4105f, group_id=92fde2624f7744c2b5e809954f75a8d9, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:54.036390 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374dd048-b4db-4dc7-8cf1-9dadf6512a38 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:54.044307 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 206/404] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 19 15:59:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:54.047699 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95538f8-341c-4618-8ed4-e78c0cff4405 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:54.048186 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95538f8-341c-4618-8ed4-e78c0cff4405 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:54.048672 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95538f8-341c-4618-8ed4-e78c0cff4405 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:54.050512 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 207/405] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:59:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 15:59:54.062417 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b7501e-8661-42c9-9202-22f746b811f8 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:54.063067 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b7501e-8661-42c9-9202-22f746b811f8 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:54.063376 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b7501e-8661-42c9-9202-22f746b811f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:54.067296 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 199/406] 10.222.0.57 () {58 vars in 1442 bytes} [Mon Sep 19 15:59:53 2022] PUT /identity/v3/projects/93b0db50dd5e4d3195beded37dd4105f/groups/92fde2624f7744c2b5e809954f75a8d9/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:59:54.068559 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374dd048-b4db-4dc7-8cf1-9dadf6512a38 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:54.102832 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-72b7501e-8661-42c9-9202-22f746b811f8 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:59:54.104051 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-72b7501e-8661-42c9-9202-22f746b811f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:59:54.215075 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:54.238428 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374dd048-b4db-4dc7-8cf1-9dadf6512a38 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:54.245608 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 208/407] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:59:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 15:59:54.298179 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1260cf3a-1d39-4e5f-8fba-6625d942f354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:54.298856 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1260cf3a-1d39-4e5f-8fba-6625d942f354 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:54.298856 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1260cf3a-1d39-4e5f-8fba-6625d942f354 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:54.298856 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1260cf3a-1d39-4e5f-8fba-6625d942f354 None admin] PATH_INFO: `/v3/groups/92fde2624f7744c2b5e809954f75a8d9` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:54.310424 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1260cf3a-1d39-4e5f-8fba-6625d942f354 None admin] RBAC: Authorizing `identity:get_group(group_id=92fde2624f7744c2b5e809954f75a8d9)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:54.310424 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1260cf3a-1d39-4e5f-8fba-6625d942f354 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:54.311325 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 200/408] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Sep 19 15:59:54 2022] GET /identity/v3/groups/92fde2624f7744c2b5e809954f75a8d9 => generated 226 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:54.318137 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b7501e-8661-42c9-9202-22f746b811f8 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:54.436852 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b7501e-8661-42c9-9202-22f746b811f8 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:54.474191 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993baedb-70cc-48d3-b385-9478ba8990ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:54.475192 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993baedb-70cc-48d3-b385-9478ba8990ed None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:54.475669 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993baedb-70cc-48d3-b385-9478ba8990ed None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:54.476199 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993baedb-70cc-48d3-b385-9478ba8990ed None admin] PATH_INFO: `/v3/projects/93b0db50dd5e4d3195beded37dd4105f` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:54.477573 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993baedb-70cc-48d3-b385-9478ba8990ed None admin] RBAC: Authorizing `identity:get_project(project_id=93b0db50dd5e4d3195beded37dd4105f)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:54.479235 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993baedb-70cc-48d3-b385-9478ba8990ed None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:54.483068 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 209/409] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 19 15:59:54 2022] GET /identity/v3/projects/93b0db50dd5e4d3195beded37dd4105f => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:54.487442 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1260cf3a-1d39-4e5f-8fba-6625d942f354 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:54.515651 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:54.608640 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993baedb-70cc-48d3-b385-9478ba8990ed None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:54.626442 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1260cf3a-1d39-4e5f-8fba-6625d942f354 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:54.682883 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b79e7ba-20c9-498b-9489-b546a63cef33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:54.683558 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b79e7ba-20c9-498b-9489-b546a63cef33 None admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:54.683838 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b79e7ba-20c9-498b-9489-b546a63cef33 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:54.684112 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b79e7ba-20c9-498b-9489-b546a63cef33 None admin] PATH_INFO: `/v3/projects/93b0db50dd5e4d3195beded37dd4105f/groups/92fde2624f7744c2b5e809954f75a8d9/roles/a1606b79a2c840238de6860bc69b49b7` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:54.693005 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b79e7ba-20c9-498b-9489-b546a63cef33 None admin] RBAC: Authorizing `identity:create_grant(project_id=93b0db50dd5e4d3195beded37dd4105f, group_id=92fde2624f7744c2b5e809954f75a8d9, role_id=a1606b79a2c840238de6860bc69b49b7)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:54.694870 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b79e7ba-20c9-498b-9489-b546a63cef33 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:54.721061 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:54.721581 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 201/410] 10.222.0.57 () {58 vars in 1442 bytes} [Mon Sep 19 15:59:54 2022] PUT /identity/v3/projects/93b0db50dd5e4d3195beded37dd4105f/groups/92fde2624f7744c2b5e809954f75a8d9/roles/a1606b79a2c840238de6860bc69b49b7 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 15:59:54.815104 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993baedb-70cc-48d3-b385-9478ba8990ed None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:54.869833 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1d530d-4b80-4915-a317-22b260ba968f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:54.870550 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1d530d-4b80-4915-a317-22b260ba968f None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:54.870833 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1d530d-4b80-4915-a317-22b260ba968f None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:54.871107 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1d530d-4b80-4915-a317-22b260ba968f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:54.888277 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1d530d-4b80-4915-a317-22b260ba968f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 15:59:54.893927 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-2c1d530d-4b80-4915-a317-22b260ba968f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 15:59:54.893927 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 15:59:54.893927 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 15:59:54.893927 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 15:59:54.893927 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 15:59:54.893927 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 15:59:54.893927 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 15:59:54.893927 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 15:59:54.893927 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 15:59:54.893927 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 15:59:54.893927 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 15:59:54.893927 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 15:59:54.893927 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 15:59:54.893927 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 15:59:54.893927 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 15:59:54.893927 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 15:59:54.894872 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 15:59:54.894872 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 15:59:54.894872 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 15:59:54.894872 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 15:59:54.894872 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 15:59:54.894872 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 15:59:54.894872 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 15:59:54.894872 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 15:59:54.894872 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 15:59:54.894872 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 15:59:54.894872 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 15:59:54.894872 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 15:59:54.894872 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 15:59:54.894872 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 15:59:54.894872 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 15:59:54.894872 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 15:59:54.895650 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 15:59:54.895650 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 15:59:54.895650 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 15:59:54.895650 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 15:59:54.895650 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 15:59:54.895650 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 15:59:54.895650 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 15:59:54.895650 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 15:59:54.895650 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 15:59:54.895650 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 15:59:54.895650 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 15:59:54.895650 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 15:59:54.895650 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 210/411] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 19 15:59:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 15:59:54.903532 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b79e7ba-20c9-498b-9489-b546a63cef33 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:54.997422 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:55.020313 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b79e7ba-20c9-498b-9489-b546a63cef33 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:55.056084 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2774be5c-36da-44d4-b28a-16862770e0e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:55.056611 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2774be5c-36da-44d4-b28a-16862770e0e9 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:55.056930 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2774be5c-36da-44d4-b28a-16862770e0e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:55.057204 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2774be5c-36da-44d4-b28a-16862770e0e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:55.057858 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2774be5c-36da-44d4-b28a-16862770e0e9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:55.058161 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2774be5c-36da-44d4-b28a-16862770e0e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:55.059539 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2774be5c-36da-44d4-b28a-16862770e0e9 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:55.068688 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 202/412] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 15:59:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:55.074518 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1d530d-4b80-4915-a317-22b260ba968f None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:55.188476 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1d530d-4b80-4915-a317-22b260ba968f None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:55.208176 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:55.237450 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f755ea76-539a-4463-8bc3-450c1a720dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:55.238095 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f755ea76-539a-4463-8bc3-450c1a720dfa None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:55.238412 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f755ea76-539a-4463-8bc3-450c1a720dfa None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:55.238666 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f755ea76-539a-4463-8bc3-450c1a720dfa None admin] PATH_INFO: `/v3/groups/716a6148123142eca9ea2581da7f2458` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:55.239801 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f755ea76-539a-4463-8bc3-450c1a720dfa None admin] RBAC: Authorizing `identity:get_group(group_id=716a6148123142eca9ea2581da7f2458)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:55.241237 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f755ea76-539a-4463-8bc3-450c1a720dfa None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:55.243213 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 211/413] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Sep 19 15:59:55 2022] GET /identity/v3/groups/716a6148123142eca9ea2581da7f2458 => generated 229 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 15:59:55.248689 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2774be5c-36da-44d4-b28a-16862770e0e9 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:55.430168 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2774be5c-36da-44d4-b28a-16862770e0e9 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:55.476320 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126d7139-ec77-4c2a-b3f2-15822d367bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:55.477313 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126d7139-ec77-4c2a-b3f2-15822d367bd4 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:55.477795 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126d7139-ec77-4c2a-b3f2-15822d367bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:55.478288 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126d7139-ec77-4c2a-b3f2-15822d367bd4 None admin] PATH_INFO: `/v3/projects/5a804037a8734a35ad3ef5ec7e34d4c6` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:55.483584 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126d7139-ec77-4c2a-b3f2-15822d367bd4 None admin] RBAC: Authorizing `identity:get_project(project_id=5a804037a8734a35ad3ef5ec7e34d4c6)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:55.485259 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126d7139-ec77-4c2a-b3f2-15822d367bd4 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:55.487158 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 203/414] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 19 15:59:55 2022] GET /identity/v3/projects/5a804037a8734a35ad3ef5ec7e34d4c6 => generated 344 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:55.492981 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f755ea76-539a-4463-8bc3-450c1a720dfa None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:55.645733 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f755ea76-539a-4463-8bc3-450c1a720dfa None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:55.705795 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ce4c66-5284-448a-86a4-ce3c9b6bfb43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:55.706515 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ce4c66-5284-448a-86a4-ce3c9b6bfb43 None admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:55.706794 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ce4c66-5284-448a-86a4-ce3c9b6bfb43 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:55.707064 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ce4c66-5284-448a-86a4-ce3c9b6bfb43 None admin] PATH_INFO: `/v3/projects/5a804037a8734a35ad3ef5ec7e34d4c6/groups/716a6148123142eca9ea2581da7f2458/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:55.709169 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ce4c66-5284-448a-86a4-ce3c9b6bfb43 None admin] RBAC: Authorizing `identity:create_grant(project_id=5a804037a8734a35ad3ef5ec7e34d4c6, group_id=716a6148123142eca9ea2581da7f2458, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:55.710724 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ce4c66-5284-448a-86a4-ce3c9b6bfb43 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:55.733691 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 212/415] 10.222.0.57 () {58 vars in 1442 bytes} [Mon Sep 19 15:59:55 2022] PUT /identity/v3/projects/5a804037a8734a35ad3ef5ec7e34d4c6/groups/716a6148123142eca9ea2581da7f2458/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:59:55.870907 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54072f7-c626-4b8c-bf7d-1006ae665717 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:55.871568 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54072f7-c626-4b8c-bf7d-1006ae665717 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:55.872062 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54072f7-c626-4b8c-bf7d-1006ae665717 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:55.873987 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 204/416] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:59:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 15:59:55.880504 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0bcbe4-3b96-4733-bc14-66afc93a9899 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:55.880828 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0bcbe4-3b96-4733-bc14-66afc93a9899 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:55.881101 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0bcbe4-3b96-4733-bc14-66afc93a9899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:55.909072 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-6d0bcbe4-3b96-4733-bc14-66afc93a9899 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:59:55.910662 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-6d0bcbe4-3b96-4733-bc14-66afc93a9899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:59:55.967319 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:56.005439 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 213/417] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:59:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 15:59:56.371465 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54072f7-c626-4b8c-bf7d-1006ae665717 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:56.383465 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:56.439761 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd7fc7e-3558-452b-ac54-7bb651b56eb6 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:56.440037 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd7fc7e-3558-452b-ac54-7bb651b56eb6 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:56.440312 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd7fc7e-3558-452b-ac54-7bb651b56eb6 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:56.442198 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 214/418] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:59:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 15:59:56.448428 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e44f006-3ebd-40c8-bb53-e4542d231d47 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:56.448703 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e44f006-3ebd-40c8-bb53-e4542d231d47 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:56.448975 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e44f006-3ebd-40c8-bb53-e4542d231d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:56.471517 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-6e44f006-3ebd-40c8-bb53-e4542d231d47 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:59:56.472710 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-6e44f006-3ebd-40c8-bb53-e4542d231d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:59:56.523460 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54072f7-c626-4b8c-bf7d-1006ae665717 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:56.600195 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7093f0-0fe0-443b-bd8d-d340550c0bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:56.601437 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7093f0-0fe0-443b-bd8d-d340550c0bdc None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:56.603203 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7093f0-0fe0-443b-bd8d-d340550c0bdc None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:56.603700 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7093f0-0fe0-443b-bd8d-d340550c0bdc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:56.615782 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 215/419] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:59:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 15:59:56.629316 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7093f0-0fe0-443b-bd8d-d340550c0bdc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 15:59:56.638939 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-ab7093f0-0fe0-443b-bd8d-d340550c0bdc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 15:59:56.638939 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 15:59:56.638939 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 15:59:56.638939 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 15:59:56.638939 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 15:59:56.638939 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 15:59:56.638939 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 15:59:56.638939 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 15:59:56.638939 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 15:59:56.638939 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 15:59:56.638939 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 15:59:56.638939 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 15:59:56.638939 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 15:59:56.638939 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 15:59:56.638939 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 15:59:56.638939 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 15:59:56.639895 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 15:59:56.639895 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 15:59:56.639895 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 15:59:56.639895 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 15:59:56.639895 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 15:59:56.639895 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 15:59:56.639895 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 15:59:56.639895 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 15:59:56.639895 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 15:59:56.639895 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 15:59:56.639895 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 15:59:56.639895 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 15:59:56.639895 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 15:59:56.639895 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 15:59:56.639895 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 15:59:56.639895 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 15:59:56.640694 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 15:59:56.640694 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 15:59:56.640694 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 15:59:56.640694 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 15:59:56.640694 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 15:59:56.640694 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 15:59:56.640694 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 15:59:56.640694 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 15:59:56.640694 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 15:59:56.640694 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 15:59:56.640694 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 15:59:56.640694 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 15:59:56.642943 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 205/420] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 15:59:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 15:59:56.648961 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e44f006-3ebd-40c8-bb53-e4542d231d47 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:56.794478 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e44f006-3ebd-40c8-bb53-e4542d231d47 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:56.796942 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:56.846724 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679201de-aa74-4d34-846c-3da8ab74db1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:56.847393 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679201de-aa74-4d34-846c-3da8ab74db1a None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:56.847676 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679201de-aa74-4d34-846c-3da8ab74db1a None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:56.847948 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679201de-aa74-4d34-846c-3da8ab74db1a None admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:56.848613 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679201de-aa74-4d34-846c-3da8ab74db1a None admin] RBAC: Adding query filter params (name=member) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:56.848916 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679201de-aa74-4d34-846c-3da8ab74db1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:56.850281 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679201de-aa74-4d34-846c-3da8ab74db1a None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:56.862098 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 216/421] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Sep 19 15:59:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:56.867910 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7093f0-0fe0-443b-bd8d-d340550c0bdc None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:56.875836 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7044540d-d662-4bee-923e-d63e03e64122 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:56.876128 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7044540d-d662-4bee-923e-d63e03e64122 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:56.876379 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7044540d-d662-4bee-923e-d63e03e64122 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:56.877959 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 217/422] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:59:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 15:59:56.883195 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3269bf6e-3242-4b5a-95d9-60518ea14b92 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:56.883413 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3269bf6e-3242-4b5a-95d9-60518ea14b92 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:56.883670 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3269bf6e-3242-4b5a-95d9-60518ea14b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:56.923860 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-3269bf6e-3242-4b5a-95d9-60518ea14b92 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:59:56.925527 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-3269bf6e-3242-4b5a-95d9-60518ea14b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:59:56.955239 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:56.993678 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7093f0-0fe0-443b-bd8d-d340550c0bdc None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:57.014553 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 218/423] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:59:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 15:59:57.016022 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3269bf6e-3242-4b5a-95d9-60518ea14b92 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:57.047965 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13b0a5f-c62c-4602-96bc-29f07655c44f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:57.048582 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b0a5f-c62c-4602-96bc-29f07655c44f None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:57.048863 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b0a5f-c62c-4602-96bc-29f07655c44f None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:57.049152 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b0a5f-c62c-4602-96bc-29f07655c44f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:57.182108 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13b0a5f-c62c-4602-96bc-29f07655c44f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:57.183651 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13b0a5f-c62c-4602-96bc-29f07655c44f None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:57.216494 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:57.236635 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3269bf6e-3242-4b5a-95d9-60518ea14b92 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:57.259657 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:57.308438 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 206/424] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 19 15:59:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 15:59:57.313420 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13b0a5f-c62c-4602-96bc-29f07655c44f None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:57.318129 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7578cb75-f3ec-45a0-b3ce-381feaaddfa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:57.319104 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7578cb75-f3ec-45a0-b3ce-381feaaddfa5 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:57.319616 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7578cb75-f3ec-45a0-b3ce-381feaaddfa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:57.320090 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7578cb75-f3ec-45a0-b3ce-381feaaddfa5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:57.332827 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7578cb75-f3ec-45a0-b3ce-381feaaddfa5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 15:59:57.338277 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-7578cb75-f3ec-45a0-b3ce-381feaaddfa5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 15:59:57.338277 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 15:59:57.338277 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 15:59:57.338277 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 15:59:57.338277 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 15:59:57.338277 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 15:59:57.338277 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 15:59:57.338277 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 15:59:57.338277 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 15:59:57.338277 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 15:59:57.338277 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 15:59:57.338277 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 15:59:57.338277 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 15:59:57.338277 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 15:59:57.338277 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 15:59:57.338277 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 15:59:57.339192 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 15:59:57.339192 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 15:59:57.339192 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 15:59:57.339192 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 15:59:57.339192 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 15:59:57.339192 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 15:59:57.339192 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 15:59:57.339192 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 15:59:57.339192 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 15:59:57.339192 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 15:59:57.339192 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 15:59:57.339192 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 15:59:57.339192 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 15:59:57.339192 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 15:59:57.339192 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 15:59:57.339192 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 15:59:57.339960 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 15:59:57.339960 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 15:59:57.339960 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 15:59:57.339960 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 15:59:57.339960 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 15:59:57.339960 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 15:59:57.339960 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 15:59:57.339960 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 15:59:57.339960 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 15:59:57.339960 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 15:59:57.339960 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 15:59:57.339960 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 15:59:57.342999 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 219/425] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Sep 19 15:59:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 15:59:57.344085 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7578cb75-f3ec-45a0-b3ce-381feaaddfa5 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:57.428897 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13b0a5f-c62c-4602-96bc-29f07655c44f None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:57.460289 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7578cb75-f3ec-45a0-b3ce-381feaaddfa5 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:57.473451 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae71a4c-e7a4-4bc5-9857-5e1be365c07c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:57.474092 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae71a4c-e7a4-4bc5-9857-5e1be365c07c None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:57.474365 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae71a4c-e7a4-4bc5-9857-5e1be365c07c None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:57.475054 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae71a4c-e7a4-4bc5-9857-5e1be365c07c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:57.490982 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae71a4c-e7a4-4bc5-9857-5e1be365c07c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 15:59:57.496467 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-0ae71a4c-e7a4-4bc5-9857-5e1be365c07c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 15:59:57.496467 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 15:59:57.496467 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 15:59:57.496467 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 15:59:57.496467 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 15:59:57.496467 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 15:59:57.496467 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 15:59:57.496467 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 15:59:57.496467 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 15:59:57.496467 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 15:59:57.496467 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 15:59:57.496467 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 15:59:57.496467 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 15:59:57.496467 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 15:59:57.496467 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 15:59:57.496467 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 15:59:57.497391 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 15:59:57.497391 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 15:59:57.497391 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 15:59:57.497391 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 15:59:57.497391 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 15:59:57.497391 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 15:59:57.497391 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 15:59:57.497391 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 15:59:57.497391 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 15:59:57.497391 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 15:59:57.497391 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 15:59:57.497391 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 15:59:57.497391 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 15:59:57.497391 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 15:59:57.497391 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 15:59:57.497391 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 15:59:57.498626 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 15:59:57.498626 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 15:59:57.498626 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 15:59:57.498626 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 15:59:57.498626 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 15:59:57.498626 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 15:59:57.498626 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 15:59:57.498626 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 15:59:57.498626 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 15:59:57.498626 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 15:59:57.498626 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 15:59:57.498626 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 15:59:57.498626 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 207/426] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 15:59:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 15:59:57.499552 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae71a4c-e7a4-4bc5-9857-5e1be365c07c None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:57.510495 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b8c006-9e8b-4086-899f-e7037df2f332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:57.511142 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b8c006-9e8b-4086-899f-e7037df2f332 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:57.511422 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b8c006-9e8b-4086-899f-e7037df2f332 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:57.511692 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b8c006-9e8b-4086-899f-e7037df2f332 None admin] PATH_INFO: `/v3/projects/ad7bddf539af4f8dabada63995dab07f` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:57.512865 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b8c006-9e8b-4086-899f-e7037df2f332 None admin] RBAC: Authorizing `identity:get_project(project_id=ad7bddf539af4f8dabada63995dab07f)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:57.514880 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b8c006-9e8b-4086-899f-e7037df2f332 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:57.517792 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 220/427] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 19 15:59:57 2022] GET /identity/v3/projects/ad7bddf539af4f8dabada63995dab07f => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:57.518823 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b8c006-9e8b-4086-899f-e7037df2f332 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:57.531038 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:57.545041 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:57.615686 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae71a4c-e7a4-4bc5-9857-5e1be365c07c None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:57.631323 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b8c006-9e8b-4086-899f-e7037df2f332 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:57.654450 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e4eb61-cbed-4073-9d41-99fbc2e99412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:57.655182 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e4eb61-cbed-4073-9d41-99fbc2e99412 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:57.655495 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e4eb61-cbed-4073-9d41-99fbc2e99412 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:57.655847 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e4eb61-cbed-4073-9d41-99fbc2e99412 None admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:57.656513 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e4eb61-cbed-4073-9d41-99fbc2e99412 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:57.656826 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e4eb61-cbed-4073-9d41-99fbc2e99412 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:57.658582 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e4eb61-cbed-4073-9d41-99fbc2e99412 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:57.667146 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2134b2-f82c-4496-a384-4e4b3aa571d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:57.667354 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 208/428] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 19 15:59:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:57.667933 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2134b2-f82c-4496-a384-4e4b3aa571d0 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:57.668081 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2134b2-f82c-4496-a384-4e4b3aa571d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:57.668157 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e4eb61-cbed-4073-9d41-99fbc2e99412 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:57.668430 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2134b2-f82c-4496-a384-4e4b3aa571d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:57.669234 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2134b2-f82c-4496-a384-4e4b3aa571d0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:57.669539 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2134b2-f82c-4496-a384-4e4b3aa571d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:57.671208 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2134b2-f82c-4496-a384-4e4b3aa571d0 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:57.682088 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 221/429] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Sep 19 15:59:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:57.687080 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2134b2-f82c-4496-a384-4e4b3aa571d0 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:57.781321 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:57.783064 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:57.792179 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e4eb61-cbed-4073-9d41-99fbc2e99412 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:57.804534 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2134b2-f82c-4496-a384-4e4b3aa571d0 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:57.826926 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cd0ce8-59a6-4831-9935-7e3dfc56d2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:57.827636 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd0ce8-59a6-4831-9935-7e3dfc56d2a6 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:57.827922 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd0ce8-59a6-4831-9935-7e3dfc56d2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:57.828199 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd0ce8-59a6-4831-9935-7e3dfc56d2a6 None admin] PATH_INFO: `/v3/groups/92fde2624f7744c2b5e809954f75a8d9` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:57.829335 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cd0ce8-59a6-4831-9935-7e3dfc56d2a6 None admin] RBAC: Authorizing `identity:get_group(group_id=92fde2624f7744c2b5e809954f75a8d9)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:57.830806 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cd0ce8-59a6-4831-9935-7e3dfc56d2a6 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:57.837293 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 209/430] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Sep 19 15:59:57 2022] GET /identity/v3/groups/92fde2624f7744c2b5e809954f75a8d9 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:57.838129 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cd0ce8-59a6-4831-9935-7e3dfc56d2a6 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:57.843948 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbd922f-5761-46d2-a684-2902e430b0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:57.844557 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd922f-5761-46d2-a684-2902e430b0d0 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:57.844837 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd922f-5761-46d2-a684-2902e430b0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:57.845107 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd922f-5761-46d2-a684-2902e430b0d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:57.917296 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbd922f-5761-46d2-a684-2902e430b0d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:57.918784 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbd922f-5761-46d2-a684-2902e430b0d0 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:57.948439 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cd0ce8-59a6-4831-9935-7e3dfc56d2a6 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:57.987562 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11082ca1-a9f4-4c7f-b381-9ae4cbda4de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:57.988176 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11082ca1-a9f4-4c7f-b381-9ae4cbda4de6 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:57.988453 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11082ca1-a9f4-4c7f-b381-9ae4cbda4de6 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:57.988724 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11082ca1-a9f4-4c7f-b381-9ae4cbda4de6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:58.006402 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:58.018227 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:58.031087 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 222/431] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 19 15:59:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 15:59:58.031087 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c00667c-f53f-4a3a-b62c-dd7920eb5f86 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:58.031087 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c00667c-f53f-4a3a-b62c-dd7920eb5f86 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:58.031087 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c00667c-f53f-4a3a-b62c-dd7920eb5f86 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:58.031087 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 223/432] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:59:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 15:59:58.031087 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c00667c-f53f-4a3a-b62c-dd7920eb5f86 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:58.070343 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11082ca1-a9f4-4c7f-b381-9ae4cbda4de6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:58.071857 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11082ca1-a9f4-4c7f-b381-9ae4cbda4de6 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:58.144830 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c00667c-f53f-4a3a-b62c-dd7920eb5f86 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:58.185202 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ac68b8-0beb-4594-8a78-120642efd5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:58.185459 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 210/433] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 19 15:59:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 15:59:58.185540 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ac68b8-0beb-4594-8a78-120642efd5c8 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:58.185607 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11082ca1-a9f4-4c7f-b381-9ae4cbda4de6 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:58.185701 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ac68b8-0beb-4594-8a78-120642efd5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:58.185701 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ac68b8-0beb-4594-8a78-120642efd5c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:58.185701 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ac68b8-0beb-4594-8a78-120642efd5c8 None admin] RBAC: Adding query filter params (group.id=92fde2624f7744c2b5e809954f75a8d9, role.id=31b0628699ea4f98824f20e7568748cc, scope.project.id=ad7bddf539af4f8dabada63995dab07f) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:58.185701 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ac68b8-0beb-4594-8a78-120642efd5c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:58.185701 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ac68b8-0beb-4594-8a78-120642efd5c8 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:58.191016 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 224/434] 10.222.0.57 () {56 vars in 1376 bytes} [Mon Sep 19 15:59:58 2022] GET /identity/v3/role_assignments?group.id=92fde2624f7744c2b5e809954f75a8d9&scope.project.id=ad7bddf539af4f8dabada63995dab07f&role.id=31b0628699ea4f98824f20e7568748cc => generated 611 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:58.198508 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0589f207-ecd1-4461-a542-e0a20f746279 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:58.198508 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0589f207-ecd1-4461-a542-e0a20f746279 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:58.198508 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0589f207-ecd1-4461-a542-e0a20f746279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:58.218472 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-0589f207-ecd1-4461-a542-e0a20f746279 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:59:58.220296 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-0589f207-ecd1-4461-a542-e0a20f746279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:59:58.268655 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:58.287478 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:58.308156 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 225/435] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:59:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 15:59:58.309696 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0589f207-ecd1-4461-a542-e0a20f746279 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:58.325082 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11082ca1-a9f4-4c7f-b381-9ae4cbda4de6 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:58.396157 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b795b77-f8af-4289-93f9-47ee13037c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:58.397244 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b795b77-f8af-4289-93f9-47ee13037c03 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:58.397892 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b795b77-f8af-4289-93f9-47ee13037c03 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:58.399294 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b795b77-f8af-4289-93f9-47ee13037c03 None admin] PATH_INFO: `/v3/projects/ad7bddf539af4f8dabada63995dab07f` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:58.403923 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b795b77-f8af-4289-93f9-47ee13037c03 None admin] RBAC: Authorizing `identity:get_project(project_id=ad7bddf539af4f8dabada63995dab07f)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:58.406139 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b795b77-f8af-4289-93f9-47ee13037c03 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:58.408165 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 211/436] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 19 15:59:58 2022] GET /identity/v3/projects/ad7bddf539af4f8dabada63995dab07f => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:58.417026 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b795b77-f8af-4289-93f9-47ee13037c03 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:58.478922 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0589f207-ecd1-4461-a542-e0a20f746279 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:58.521759 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35b1e88-206c-48ed-a05c-8444e8658b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:58.522693 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b1e88-206c-48ed-a05c-8444e8658b47 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:58.522976 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b1e88-206c-48ed-a05c-8444e8658b47 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:58.523416 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b1e88-206c-48ed-a05c-8444e8658b47 None admin] PATH_INFO: `/v3/projects/93b0db50dd5e4d3195beded37dd4105f` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:58.524948 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35b1e88-206c-48ed-a05c-8444e8658b47 None admin] RBAC: Authorizing `identity:get_project(project_id=93b0db50dd5e4d3195beded37dd4105f)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:58.526988 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35b1e88-206c-48ed-a05c-8444e8658b47 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:58.529747 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 226/437] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 19 15:59:58 2022] GET /identity/v3/projects/93b0db50dd5e4d3195beded37dd4105f => generated 302 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:58.530148 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b795b77-f8af-4289-93f9-47ee13037c03 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:58.530981 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483797d3-095f-4464-a56d-52e65abe3884 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:58.531261 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483797d3-095f-4464-a56d-52e65abe3884 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:58.531534 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483797d3-095f-4464-a56d-52e65abe3884 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:58.533011 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 227/438] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 15:59:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 15:59:58.537203 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483797d3-095f-4464-a56d-52e65abe3884 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:58.565819 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:58.576683 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9584f3d1-debf-4f10-ae93-3901c316c80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:58.577324 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9584f3d1-debf-4f10-ae93-3901c316c80f None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:58.577603 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9584f3d1-debf-4f10-ae93-3901c316c80f None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:58.577873 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9584f3d1-debf-4f10-ae93-3901c316c80f None admin] PATH_INFO: `/v3/groups/92fde2624f7744c2b5e809954f75a8d9` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:58.579135 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9584f3d1-debf-4f10-ae93-3901c316c80f None admin] RBAC: Authorizing `identity:get_group(group_id=92fde2624f7744c2b5e809954f75a8d9)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:58.580557 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9584f3d1-debf-4f10-ae93-3901c316c80f None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:58.582517 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 212/439] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Sep 19 15:59:58 2022] GET /identity/v3/groups/92fde2624f7744c2b5e809954f75a8d9 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:58.583656 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e953b070-44a8-4cd9-8eee-dd428a76a58b None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:58.583935 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e953b070-44a8-4cd9-8eee-dd428a76a58b None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:58.584206 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e953b070-44a8-4cd9-8eee-dd428a76a58b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:58.606749 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:58.608010 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-e953b070-44a8-4cd9-8eee-dd428a76a58b None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:59:58.609236 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-e953b070-44a8-4cd9-8eee-dd428a76a58b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:59:58.670949 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483797d3-095f-4464-a56d-52e65abe3884 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:58.691279 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 213/440] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 15:59:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 15:59:58.692192 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e953b070-44a8-4cd9-8eee-dd428a76a58b None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:58.725304 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab28ed2-1145-4f6c-a46a-c053ea9c9454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:58.725950 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab28ed2-1145-4f6c-a46a-c053ea9c9454 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:58.726223 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab28ed2-1145-4f6c-a46a-c053ea9c9454 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:58.726516 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab28ed2-1145-4f6c-a46a-c053ea9c9454 None admin] PATH_INFO: `/v3/groups/92fde2624f7744c2b5e809954f75a8d9` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:58.727982 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab28ed2-1145-4f6c-a46a-c053ea9c9454 None admin] RBAC: Authorizing `identity:get_group(group_id=92fde2624f7744c2b5e809954f75a8d9)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:58.729406 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab28ed2-1145-4f6c-a46a-c053ea9c9454 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:58.731572 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 228/441] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Sep 19 15:59:58 2022] GET /identity/v3/groups/92fde2624f7744c2b5e809954f75a8d9 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:58.732712 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab28ed2-1145-4f6c-a46a-c053ea9c9454 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:58.843888 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e953b070-44a8-4cd9-8eee-dd428a76a58b None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:58.866307 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab28ed2-1145-4f6c-a46a-c053ea9c9454 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:58.884355 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:58.889371 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:58.902518 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96afb565-6346-4b01-b9b7-f748da92fe07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:58.903234 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96afb565-6346-4b01-b9b7-f748da92fe07 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:58.903579 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96afb565-6346-4b01-b9b7-f748da92fe07 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:58.903918 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96afb565-6346-4b01-b9b7-f748da92fe07 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:58.904756 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96afb565-6346-4b01-b9b7-f748da92fe07 None admin] RBAC: Adding query filter params (group.id=92fde2624f7744c2b5e809954f75a8d9, role.id=a1606b79a2c840238de6860bc69b49b7, scope.project.id=ad7bddf539af4f8dabada63995dab07f) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:58.905158 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96afb565-6346-4b01-b9b7-f748da92fe07 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:58.907071 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96afb565-6346-4b01-b9b7-f748da92fe07 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:58.918408 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 214/442] 10.222.0.57 () {56 vars in 1376 bytes} [Mon Sep 19 15:59:58 2022] GET /identity/v3/role_assignments?group.id=92fde2624f7744c2b5e809954f75a8d9&scope.project.id=ad7bddf539af4f8dabada63995dab07f&role.id=a1606b79a2c840238de6860bc69b49b7 => generated 611 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:58.918408 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96afb565-6346-4b01-b9b7-f748da92fe07 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:58.925177 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bedc202-e201-4c52-a278-f13489cb1456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:58.925177 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bedc202-e201-4c52-a278-f13489cb1456 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:58.925177 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bedc202-e201-4c52-a278-f13489cb1456 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:58.925177 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bedc202-e201-4c52-a278-f13489cb1456 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:58.941165 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bedc202-e201-4c52-a278-f13489cb1456 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 15:59:58.946848 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-9bedc202-e201-4c52-a278-f13489cb1456 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 15:59:58.946848 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 15:59:58.946848 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 15:59:58.946848 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 15:59:58.946848 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 15:59:58.946848 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 15:59:58.946848 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 15:59:58.946848 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 15:59:58.946848 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 15:59:58.946848 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 15:59:58.946848 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 15:59:58.946848 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 15:59:58.946848 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 15:59:58.946848 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 15:59:58.946848 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 15:59:58.946848 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 15:59:58.948003 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 15:59:58.948003 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 15:59:58.948003 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 15:59:58.948003 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 15:59:58.948003 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 15:59:58.948003 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 15:59:58.948003 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 15:59:58.948003 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 15:59:58.948003 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 15:59:58.948003 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 15:59:58.948003 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 15:59:58.948003 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 15:59:58.948003 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 15:59:58.948003 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 15:59:58.948003 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 15:59:58.948003 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 15:59:58.948838 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 15:59:58.948838 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 15:59:58.948838 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 15:59:58.948838 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 15:59:58.948838 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 15:59:58.948838 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 15:59:58.948838 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 15:59:58.948838 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 15:59:58.948838 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 15:59:58.948838 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 15:59:58.948838 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 15:59:58.948838 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 15:59:58.949436 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 229/443] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Sep 19 15:59:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 15:59:58.954792 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bedc202-e201-4c52-a278-f13489cb1456 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:59.075387 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96afb565-6346-4b01-b9b7-f748da92fe07 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:59.093378 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bedc202-e201-4c52-a278-f13489cb1456 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:59.115190 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fa97c1-c9ff-4b5e-b33c-2ff2ac8e761f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:59.115689 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fa97c1-c9ff-4b5e-b33c-2ff2ac8e761f None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:59.115967 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fa97c1-c9ff-4b5e-b33c-2ff2ac8e761f None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:59.116233 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fa97c1-c9ff-4b5e-b33c-2ff2ac8e761f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:59.116961 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fa97c1-c9ff-4b5e-b33c-2ff2ac8e761f None admin] RBAC: Adding query filter params (group.id=92fde2624f7744c2b5e809954f75a8d9, role.id=31b0628699ea4f98824f20e7568748cc, scope.project.id=93b0db50dd5e4d3195beded37dd4105f) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:59.117899 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fa97c1-c9ff-4b5e-b33c-2ff2ac8e761f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:59.118726 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fa97c1-c9ff-4b5e-b33c-2ff2ac8e761f None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:59.125262 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 215/444] 10.222.0.57 () {56 vars in 1376 bytes} [Mon Sep 19 15:59:58 2022] GET /identity/v3/role_assignments?group.id=92fde2624f7744c2b5e809954f75a8d9&scope.project.id=93b0db50dd5e4d3195beded37dd4105f&role.id=31b0628699ea4f98824f20e7568748cc => generated 611 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:59.126359 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fa97c1-c9ff-4b5e-b33c-2ff2ac8e761f None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:59.150138 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e91415-b5a2-4fea-a339-18816650b646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:59.151356 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e91415-b5a2-4fea-a339-18816650b646 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:59.152486 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e91415-b5a2-4fea-a339-18816650b646 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:59.152879 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e91415-b5a2-4fea-a339-18816650b646 None admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:59.153703 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e91415-b5a2-4fea-a339-18816650b646 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:59.154554 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e91415-b5a2-4fea-a339-18816650b646 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:59.156709 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e91415-b5a2-4fea-a339-18816650b646 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:59.165900 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 230/445] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 19 15:59:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 15:59:59.172910 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e91415-b5a2-4fea-a339-18816650b646 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:59.198920 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:59.238026 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:59.276883 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fa97c1-c9ff-4b5e-b33c-2ff2ac8e761f None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:59.325210 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd2c9d5-98f3-44b0-a8f6-c4d472d10616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:59.326068 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd2c9d5-98f3-44b0-a8f6-c4d472d10616 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:59.326484 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd2c9d5-98f3-44b0-a8f6-c4d472d10616 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:59.326958 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd2c9d5-98f3-44b0-a8f6-c4d472d10616 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:59.334915 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e91415-b5a2-4fea-a339-18816650b646 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:59.342308 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd2c9d5-98f3-44b0-a8f6-c4d472d10616 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 15:59:59.348098 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-dcd2c9d5-98f3-44b0-a8f6-c4d472d10616 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 15:59:59.348098 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 15:59:59.348098 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 15:59:59.348098 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 15:59:59.348098 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 15:59:59.348098 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 15:59:59.348098 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 15:59:59.348098 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 15:59:59.348098 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 15:59:59.348098 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 15:59:59.348098 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 15:59:59.348098 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 15:59:59.348098 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 15:59:59.348098 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 15:59:59.348098 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 15:59:59.348098 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 15:59:59.349170 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 15:59:59.349170 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 15:59:59.349170 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 15:59:59.349170 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 15:59:59.349170 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 15:59:59.349170 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 15:59:59.349170 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 15:59:59.349170 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 15:59:59.349170 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 15:59:59.349170 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 15:59:59.349170 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 15:59:59.349170 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 15:59:59.349170 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 15:59:59.349170 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 15:59:59.349170 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 15:59:59.349170 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 15:59:59.350122 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 15:59:59.350122 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 15:59:59.350122 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 15:59:59.350122 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 15:59:59.350122 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 15:59:59.350122 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 15:59:59.350122 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 15:59:59.350122 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 15:59:59.350122 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 15:59:59.350122 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 15:59:59.350122 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 15:59:59.350122 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 15:59:59.352082 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 216/446] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 19 15:59:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 15:59:59.358589 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd2c9d5-98f3-44b0-a8f6-c4d472d10616 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:59.380635 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a2f1bf-db8d-4f22-9455-3ccee92b1795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:59.382141 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2f1bf-db8d-4f22-9455-3ccee92b1795 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:59.382141 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2f1bf-db8d-4f22-9455-3ccee92b1795 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:59.382141 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2f1bf-db8d-4f22-9455-3ccee92b1795 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:59.464942 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a2f1bf-db8d-4f22-9455-3ccee92b1795 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:59.466450 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a2f1bf-db8d-4f22-9455-3ccee92b1795 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:59.497246 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd2c9d5-98f3-44b0-a8f6-c4d472d10616 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:59.505401 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:59.512203 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:59.539690 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c97b9e-b7e5-47e0-8b75-93556c0c6088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:59.540335 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c97b9e-b7e5-47e0-8b75-93556c0c6088 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:59.540578 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c97b9e-b7e5-47e0-8b75-93556c0c6088 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:59.540851 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c97b9e-b7e5-47e0-8b75-93556c0c6088 None admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:59.541784 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c97b9e-b7e5-47e0-8b75-93556c0c6088 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:59:59.542090 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c97b9e-b7e5-47e0-8b75-93556c0c6088 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:59.543488 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c97b9e-b7e5-47e0-8b75-93556c0c6088 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:59.552997 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 217/447] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 15:59:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:59.560924 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c97b9e-b7e5-47e0-8b75-93556c0c6088 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:59.587115 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 231/448] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 19 15:59:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 19 15:59:59.595317 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a2f1bf-db8d-4f22-9455-3ccee92b1795 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:59.670404 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c97b9e-b7e5-47e0-8b75-93556c0c6088 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:59.699043 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a2f1bf-db8d-4f22-9455-3ccee92b1795 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:59.703286 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667b303b-d103-4037-b362-b65c76ae8963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:59.703905 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667b303b-d103-4037-b362-b65c76ae8963 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:59.704180 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667b303b-d103-4037-b362-b65c76ae8963 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:59.704449 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667b303b-d103-4037-b362-b65c76ae8963 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:59.732699 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f5e9a3-0f45-4e4f-91c2-be86aa392afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:59.733327 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f5e9a3-0f45-4e4f-91c2-be86aa392afc None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:59.733604 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f5e9a3-0f45-4e4f-91c2-be86aa392afc None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:59.733876 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f5e9a3-0f45-4e4f-91c2-be86aa392afc None admin] PATH_INFO: `/v3/projects/93b0db50dd5e4d3195beded37dd4105f` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:59.734994 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f5e9a3-0f45-4e4f-91c2-be86aa392afc None admin] RBAC: Authorizing `identity:get_project(project_id=93b0db50dd5e4d3195beded37dd4105f)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:59.736448 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f5e9a3-0f45-4e4f-91c2-be86aa392afc None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:59.738091 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 232/449] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 19 15:59:59 2022] GET /identity/v3/projects/93b0db50dd5e4d3195beded37dd4105f => generated 302 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:59:59.743565 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f5e9a3-0f45-4e4f-91c2-be86aa392afc None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:59.746894 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:59.747126 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:59.775013 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667b303b-d103-4037-b362-b65c76ae8963 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:59.776583 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667b303b-d103-4037-b362-b65c76ae8963 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:59.848739 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f5e9a3-0f45-4e4f-91c2-be86aa392afc None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:59.874999 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 218/450] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 19 15:59:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 19 15:59:59.880748 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667b303b-d103-4037-b362-b65c76ae8963 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:59.882071 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba67d87d-34b9-4c48-94d1-570f8aa18941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:59:59.882729 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba67d87d-34b9-4c48-94d1-570f8aa18941 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:59:59.883006 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba67d87d-34b9-4c48-94d1-570f8aa18941 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:59:59.883281 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba67d87d-34b9-4c48-94d1-570f8aa18941 None admin] PATH_INFO: `/v3/groups/92fde2624f7744c2b5e809954f75a8d9` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:59:59.884427 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba67d87d-34b9-4c48-94d1-570f8aa18941 None admin] RBAC: Authorizing `identity:get_group(group_id=92fde2624f7744c2b5e809954f75a8d9)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:59:59.885860 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba67d87d-34b9-4c48-94d1-570f8aa18941 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:59:59.887439 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 233/451] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Sep 19 15:59:59 2022] GET /identity/v3/groups/92fde2624f7744c2b5e809954f75a8d9 => generated 226 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 15:59:59.893955 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba67d87d-34b9-4c48-94d1-570f8aa18941 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:59:59.954437 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:59.963774 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:59:59.980162 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667b303b-d103-4037-b362-b65c76ae8963 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:59:59.995648 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba67d87d-34b9-4c48-94d1-570f8aa18941 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:00.025061 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991a3af1-8dd3-4a5b-a762-4269a0b5da28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:00.025803 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991a3af1-8dd3-4a5b-a762-4269a0b5da28 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:00.026080 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991a3af1-8dd3-4a5b-a762-4269a0b5da28 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:00.026349 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991a3af1-8dd3-4a5b-a762-4269a0b5da28 None admin] PATH_INFO: `/v3/projects/5a804037a8734a35ad3ef5ec7e34d4c6` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:00.027472 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991a3af1-8dd3-4a5b-a762-4269a0b5da28 None admin] RBAC: Authorizing `identity:get_project(project_id=5a804037a8734a35ad3ef5ec7e34d4c6)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:00.028897 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991a3af1-8dd3-4a5b-a762-4269a0b5da28 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:00.029745 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc6147a-0151-467b-b946-2bca124f077c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:00.030346 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc6147a-0151-467b-b946-2bca124f077c None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:00.030556 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 219/452] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 19 15:59:59 2022] GET /identity/v3/projects/5a804037a8734a35ad3ef5ec7e34d4c6 => generated 344 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:00:00.030653 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc6147a-0151-467b-b946-2bca124f077c None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:00.030924 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc6147a-0151-467b-b946-2bca124f077c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:00.031736 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc6147a-0151-467b-b946-2bca124f077c None admin] RBAC: Adding query filter params (group.id=92fde2624f7744c2b5e809954f75a8d9, role.id=a1606b79a2c840238de6860bc69b49b7, scope.project.id=93b0db50dd5e4d3195beded37dd4105f) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:00.032069 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc6147a-0151-467b-b946-2bca124f077c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:00.033499 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc6147a-0151-467b-b946-2bca124f077c None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:00.036156 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991a3af1-8dd3-4a5b-a762-4269a0b5da28 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:00.039594 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 234/453] 10.222.0.57 () {56 vars in 1376 bytes} [Mon Sep 19 15:59:59 2022] GET /identity/v3/role_assignments?group.id=92fde2624f7744c2b5e809954f75a8d9&scope.project.id=93b0db50dd5e4d3195beded37dd4105f&role.id=a1606b79a2c840238de6860bc69b49b7 => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:00:00.147492 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991a3af1-8dd3-4a5b-a762-4269a0b5da28 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:00.183160 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf7d635-4ee3-4ae5-bd6e-8cf012163e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:00.183810 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf7d635-4ee3-4ae5-bd6e-8cf012163e94 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:00.184090 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf7d635-4ee3-4ae5-bd6e-8cf012163e94 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:00.184357 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf7d635-4ee3-4ae5-bd6e-8cf012163e94 None admin] PATH_INFO: `/v3/groups/716a6148123142eca9ea2581da7f2458` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:00.186234 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf7d635-4ee3-4ae5-bd6e-8cf012163e94 None admin] RBAC: Authorizing `identity:get_group(group_id=716a6148123142eca9ea2581da7f2458)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:00.187778 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf7d635-4ee3-4ae5-bd6e-8cf012163e94 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:00.189416 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 220/454] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Sep 19 16:00:00 2022] GET /identity/v3/groups/716a6148123142eca9ea2581da7f2458 => generated 229 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:00:00.195573 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc6147a-0151-467b-b946-2bca124f077c None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:00.301292 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc6147a-0151-467b-b946-2bca124f077c None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:00.338576 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b3e934-065f-4c13-80d7-c77ec32c6bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:00.339182 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b3e934-065f-4c13-80d7-c77ec32c6bcc None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:00.339462 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b3e934-065f-4c13-80d7-c77ec32c6bcc None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:00.339734 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b3e934-065f-4c13-80d7-c77ec32c6bcc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:00.340462 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b3e934-065f-4c13-80d7-c77ec32c6bcc None admin] RBAC: Adding query filter params (group.id=716a6148123142eca9ea2581da7f2458, role.id=63d2d197bfd6497583e94bfb065ce29b, scope.project.id=5a804037a8734a35ad3ef5ec7e34d4c6) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:00.340769 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b3e934-065f-4c13-80d7-c77ec32c6bcc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:00.342998 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b3e934-065f-4c13-80d7-c77ec32c6bcc None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:00.345942 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:00.349980 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 235/455] 10.222.0.57 () {56 vars in 1376 bytes} [Mon Sep 19 16:00:00 2022] GET /identity/v3/role_assignments?group.id=716a6148123142eca9ea2581da7f2458&scope.project.id=5a804037a8734a35ad3ef5ec7e34d4c6&role.id=63d2d197bfd6497583e94bfb065ce29b => generated 611 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:02.974091 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b1ebed-fe8f-456f-927a-9c5aaaa1272e None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:02.974957 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b1ebed-fe8f-456f-927a-9c5aaaa1272e None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:02.975376 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b1ebed-fe8f-456f-927a-9c5aaaa1272e None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:02.977122 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 221/456] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:00:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:00:02.982753 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14fc322-b69d-49c8-9aed-fa9cc6dd538e None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:02.983578 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14fc322-b69d-49c8-9aed-fa9cc6dd538e None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:02.984002 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14fc322-b69d-49c8-9aed-fa9cc6dd538e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:03.011411 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-c14fc322-b69d-49c8-9aed-fa9cc6dd538e None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:00:03.012881 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-c14fc322-b69d-49c8-9aed-fa9cc6dd538e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:00:03.103209 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 236/457] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:00:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 16:00:03.128151 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc33ec-99a0-475b-b473-ab7eea457a70 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:03.128661 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc33ec-99a0-475b-b473-ab7eea457a70 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:03.129085 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc33ec-99a0-475b-b473-ab7eea457a70 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:03.130589 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 222/458] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:00:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:00:03.136061 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a66bc3-f3cf-4314-9f4e-88e829d3d1a0 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:03.136543 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a66bc3-f3cf-4314-9f4e-88e829d3d1a0 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:03.136963 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a66bc3-f3cf-4314-9f4e-88e829d3d1a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:03.168410 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-66a66bc3-f3cf-4314-9f4e-88e829d3d1a0 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:00:03.171577 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-66a66bc3-f3cf-4314-9f4e-88e829d3d1a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:00:03.274702 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 237/459] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:00:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 16:00:03.283221 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44ad96e-2dd9-4a45-a21e-a184b0fe6b97 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:03.283500 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44ad96e-2dd9-4a45-a21e-a184b0fe6b97 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:03.283768 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44ad96e-2dd9-4a45-a21e-a184b0fe6b97 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:03.285347 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 223/460] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:00:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:00:03.290685 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f72307-b4d0-4e30-957d-d1106ecb4aa5 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:03.291439 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f72307-b4d0-4e30-957d-d1106ecb4aa5 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:03.291881 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f72307-b4d0-4e30-957d-d1106ecb4aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:03.316030 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-89f72307-b4d0-4e30-957d-d1106ecb4aa5 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:00:03.317441 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-89f72307-b4d0-4e30-957d-d1106ecb4aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:00:03.394495 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:03.411688 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 238/461] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:00:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 16:00:03.450459 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44ad96e-2dd9-4a45-a21e-a184b0fe6b97 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:03.470714 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:03.609051 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44ad96e-2dd9-4a45-a21e-a184b0fe6b97 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:03.668528 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd88fb75-5707-4d93-9cbf-72e452c6a0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:03.669091 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd88fb75-5707-4d93-9cbf-72e452c6a0c4 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:03.669391 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd88fb75-5707-4d93-9cbf-72e452c6a0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:03.669701 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd88fb75-5707-4d93-9cbf-72e452c6a0c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:03.687948 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd88fb75-5707-4d93-9cbf-72e452c6a0c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:03.689633 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd88fb75-5707-4d93-9cbf-72e452c6a0c4 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:03.692966 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f72307-b4d0-4e30-957d-d1106ecb4aa5 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:03.712142 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-dd88fb75-5707-4d93-9cbf-72e452c6a0c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:03.712142 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:03.712142 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:03.712142 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:03.712142 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:03.712142 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:03.712142 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:03.712142 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:03.712142 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:03.712142 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:03.712142 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:03.712142 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:03.712142 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:00:03.712142 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:00:03.712142 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:00:03.712142 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:00:03.713093 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:03.713093 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:03.713093 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:03.713093 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:03.713093 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:03.713093 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:03.713093 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:03.713093 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:03.713093 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:03.713093 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:03.713093 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:03.713093 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:03.713093 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:03.713093 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:03.713093 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:03.713093 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:03.713093 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:00:03.713928 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:00:03.713928 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:03.713928 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:03.713928 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 224/462] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:00:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:03.717727 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd88fb75-5707-4d93-9cbf-72e452c6a0c4 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:03.798743 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:03.840037 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd88fb75-5707-4d93-9cbf-72e452c6a0c4 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:03.840288 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f72307-b4d0-4e30-957d-d1106ecb4aa5 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:03.880290 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7e65cb-22aa-4fb5-b331-20f12ce381ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:03.881229 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e65cb-22aa-4fb5-b331-20f12ce381ee None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:03.881767 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e65cb-22aa-4fb5-b331-20f12ce381ee None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:03.882417 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e65cb-22aa-4fb5-b331-20f12ce381ee None admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:03.884060 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7e65cb-22aa-4fb5-b331-20f12ce381ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:03.884730 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7e65cb-22aa-4fb5-b331-20f12ce381ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:03.886304 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7e65cb-22aa-4fb5-b331-20f12ce381ee None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:03.888976 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ed8790-0f7e-44e0-a366-a6e1813af6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:03.889988 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed8790-0f7e-44e0-a366-a6e1813af6eb None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:03.890390 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed8790-0f7e-44e0-a366-a6e1813af6eb None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:03.890764 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed8790-0f7e-44e0-a366-a6e1813af6eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:03.903899 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ed8790-0f7e-44e0-a366-a6e1813af6eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:03.906964 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ed8790-0f7e-44e0-a366-a6e1813af6eb None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:03.909111 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 225/463] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:00:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:03.910297 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7e65cb-22aa-4fb5-b331-20f12ce381ee None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:03.934600 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-63ed8790-0f7e-44e0-a366-a6e1813af6eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:03.934600 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:03.934600 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:03.934600 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:03.934600 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:03.934600 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:03.934600 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:03.934600 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:03.934600 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:03.934600 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:03.934600 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:03.934600 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:03.934600 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:00:03.934600 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:00:03.934600 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:00:03.934600 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:00:03.935583 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:03.935583 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:03.935583 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:03.935583 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:03.935583 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:03.935583 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:03.935583 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:03.935583 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:03.935583 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:03.935583 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:03.935583 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:03.935583 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:03.935583 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:03.935583 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:03.935583 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:03.935583 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:03.935583 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:00:03.936418 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:00:03.936418 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:03.936418 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:03.938726 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 239/464] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:00:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:03.940098 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ed8790-0f7e-44e0-a366-a6e1813af6eb None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:03.992223 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:04.048099 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7e65cb-22aa-4fb5-b331-20f12ce381ee None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:04.059818 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ed8790-0f7e-44e0-a366-a6e1813af6eb None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:04.076810 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:04.089768 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac667bbb-2410-4dac-abd6-806dd103e5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:04.090737 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac667bbb-2410-4dac-abd6-806dd103e5e4 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:04.091293 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac667bbb-2410-4dac-abd6-806dd103e5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:04.091771 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac667bbb-2410-4dac-abd6-806dd103e5e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:04.102169 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac667bbb-2410-4dac-abd6-806dd103e5e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:04.102308 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade5b09a-70d1-4fd6-b226-558b70531ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:04.102932 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade5b09a-70d1-4fd6-b226-558b70531ec6 None admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:04.103212 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade5b09a-70d1-4fd6-b226-558b70531ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:04.103486 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade5b09a-70d1-4fd6-b226-558b70531ec6 None admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:04.103637 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac667bbb-2410-4dac-abd6-806dd103e5e4 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:04.104208 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade5b09a-70d1-4fd6-b226-558b70531ec6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:04.105581 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade5b09a-70d1-4fd6-b226-558b70531ec6 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:04.119164 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-ac667bbb-2410-4dac-abd6-806dd103e5e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:04.119164 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:04.119164 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:04.119164 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:04.119164 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:04.119164 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:04.119164 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:04.119164 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:04.119164 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:04.119164 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:04.119164 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:04.119164 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:04.119164 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:00:04.119164 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:00:04.119164 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:00:04.119164 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:00:04.120321 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:04.120321 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:04.120321 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:04.120321 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:04.120321 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:04.120321 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:04.120321 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:04.120321 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:04.120321 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:04.120321 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:04.120321 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:04.120321 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:04.120321 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:04.120321 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:04.120321 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:04.120321 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:04.120321 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:00:04.121139 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:00:04.121139 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:04.121139 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:04.121139 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 226/465] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:00:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:04.121139 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac667bbb-2410-4dac-abd6-806dd103e5e4 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:04.134668 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 240/466] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Sep 19 16:00:03 2022] POST /identity/v3/users => generated 246 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:00:04.136512 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5443607a-c93b-49bf-bd21-b3a71a17e156 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:04.136994 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5443607a-c93b-49bf-bd21-b3a71a17e156 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:04.137461 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5443607a-c93b-49bf-bd21-b3a71a17e156 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:04.140042 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 241/467] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:00:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:00:04.141255 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5443607a-c93b-49bf-bd21-b3a71a17e156 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:04.250265 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac667bbb-2410-4dac-abd6-806dd103e5e4 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:04.255986 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:04.262670 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5443607a-c93b-49bf-bd21-b3a71a17e156 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:04.296018 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238b39d3-17c4-41f5-a98d-168b2aee2e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:04.296653 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b39d3-17c4-41f5-a98d-168b2aee2e58 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:04.296935 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b39d3-17c4-41f5-a98d-168b2aee2e58 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:04.297212 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b39d3-17c4-41f5-a98d-168b2aee2e58 None admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:04.297874 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238b39d3-17c4-41f5-a98d-168b2aee2e58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:04.298247 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238b39d3-17c4-41f5-a98d-168b2aee2e58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:04.300096 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238b39d3-17c4-41f5-a98d-168b2aee2e58 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:04.305299 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d49b8-cc27-4943-bf7a-eb98a762badc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:04.306238 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d49b8-cc27-4943-bf7a-eb98a762badc None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:04.306736 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d49b8-cc27-4943-bf7a-eb98a762badc None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:04.307202 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d49b8-cc27-4943-bf7a-eb98a762badc None admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:04.308068 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53d49b8-cc27-4943-bf7a-eb98a762badc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:04.308580 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53d49b8-cc27-4943-bf7a-eb98a762badc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:04.310717 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53d49b8-cc27-4943-bf7a-eb98a762badc None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:04.314950 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 227/468] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:00:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:04.316251 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd868f8a-c401-4120-ac4d-2540e19fa6b9 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:04.316536 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd868f8a-c401-4120-ac4d-2540e19fa6b9 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:04.316813 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd868f8a-c401-4120-ac4d-2540e19fa6b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:04.325455 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 242/469] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:00:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:04.327154 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d49b8-cc27-4943-bf7a-eb98a762badc None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:04.341616 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-bd868f8a-c401-4120-ac4d-2540e19fa6b9 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:00:04.343416 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-bd868f8a-c401-4120-ac4d-2540e19fa6b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:00:04.378003 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:04.433418 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 228/470] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:00:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 16:00:04.434405 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd868f8a-c401-4120-ac4d-2540e19fa6b9 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:04.453870 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d49b8-cc27-4943-bf7a-eb98a762badc None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:04.491971 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026ab790-ced4-4684-bd31-f9557017406b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:04.492857 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026ab790-ced4-4684-bd31-f9557017406b None admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:04.492857 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026ab790-ced4-4684-bd31-f9557017406b None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:04.493109 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026ab790-ced4-4684-bd31-f9557017406b None admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:04.493750 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026ab790-ced4-4684-bd31-f9557017406b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:04.495214 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026ab790-ced4-4684-bd31-f9557017406b None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:04.527752 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 243/471] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Sep 19 16:00:04 2022] POST /identity/v3/users => generated 248 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:00:04.569462 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd868f8a-c401-4120-ac4d-2540e19fa6b9 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:04.610838 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0845cff8-8f73-4ec7-a93e-1eb5b79741f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:04.611822 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0845cff8-8f73-4ec7-a93e-1eb5b79741f3 None admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:04.612323 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0845cff8-8f73-4ec7-a93e-1eb5b79741f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:04.612808 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0845cff8-8f73-4ec7-a93e-1eb5b79741f3 None admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:04.613651 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0845cff8-8f73-4ec7-a93e-1eb5b79741f3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:04.615309 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0845cff8-8f73-4ec7-a93e-1eb5b79741f3 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:04.647217 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 229/472] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Sep 19 16:00:04 2022] POST /identity/v3/users => generated 248 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:00:04.860883 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026ab790-ced4-4684-bd31-f9557017406b None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:04.880472 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:04.944722 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b4d936-fd64-499b-8e13-ea753b4d4c19 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:04.945697 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b4d936-fd64-499b-8e13-ea753b4d4c19 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:04.945979 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b4d936-fd64-499b-8e13-ea753b4d4c19 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:04.947591 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 230/473] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:00:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 16:00:04.953799 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98f09e6-4bc1-453a-9f73-d0bbef5376ca None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:04.954354 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98f09e6-4bc1-453a-9f73-d0bbef5376ca None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:04.954868 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98f09e6-4bc1-453a-9f73-d0bbef5376ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:04.980356 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-f98f09e6-4bc1-453a-9f73-d0bbef5376ca None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:00:04.981614 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-f98f09e6-4bc1-453a-9f73-d0bbef5376ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:00:04.990762 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026ab790-ced4-4684-bd31-f9557017406b None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:05.003356 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:05.037934 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406b331c-697a-4ea6-b390-af79212af20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:05.038621 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b331c-697a-4ea6-b390-af79212af20c None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:05.038902 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b331c-697a-4ea6-b390-af79212af20c None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:05.039174 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b331c-697a-4ea6-b390-af79212af20c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:05.056029 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406b331c-697a-4ea6-b390-af79212af20c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:05.057575 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406b331c-697a-4ea6-b390-af79212af20c None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:05.076696 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 231/474] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:00:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 16:00:05.077300 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-406b331c-697a-4ea6-b390-af79212af20c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:05.077300 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:05.077300 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:05.077300 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:05.077300 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:05.077300 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:05.077300 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:05.077300 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:05.077300 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:05.077300 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:05.077300 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:05.077300 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:05.077300 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:00:05.077300 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:00:05.077300 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:00:05.077300 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:00:05.078133 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:05.078133 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:05.078133 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:05.078133 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:05.078133 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:05.078133 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:05.078133 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:05.078133 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:05.078133 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:05.078133 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:05.078133 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:05.078133 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:05.078133 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:05.078133 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:05.078133 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:05.078133 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:05.078133 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:00:05.078990 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:00:05.078990 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:05.078990 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:05.082412 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 244/475] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:00:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:05.084922 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98f09e6-4bc1-453a-9f73-d0bbef5376ca None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:05.193589 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98f09e6-4bc1-453a-9f73-d0bbef5376ca None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:05.229769 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ae60da-23bb-4040-9ed7-1a849202d1ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:05.230385 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ae60da-23bb-4040-9ed7-1a849202d1ee None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:05.230722 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ae60da-23bb-4040-9ed7-1a849202d1ee None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:05.231003 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ae60da-23bb-4040-9ed7-1a849202d1ee None admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:05.231652 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ae60da-23bb-4040-9ed7-1a849202d1ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:05.232038 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ae60da-23bb-4040-9ed7-1a849202d1ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:05.233436 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ae60da-23bb-4040-9ed7-1a849202d1ee None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:05.245776 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 232/476] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:00:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:05.251928 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406b331c-697a-4ea6-b390-af79212af20c None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:05.330016 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:05.367732 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406b331c-697a-4ea6-b390-af79212af20c None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:05.375132 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ae60da-23bb-4040-9ed7-1a849202d1ee None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:05.392044 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:05.426071 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbf27d5-27d1-4f6f-b960-5760b57b110d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:05.427053 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbf27d5-27d1-4f6f-b960-5760b57b110d None admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:05.428482 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbf27d5-27d1-4f6f-b960-5760b57b110d None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:05.431544 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbf27d5-27d1-4f6f-b960-5760b57b110d None admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:05.432499 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbf27d5-27d1-4f6f-b960-5760b57b110d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:05.434153 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbf27d5-27d1-4f6f-b960-5760b57b110d None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:05.469272 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 245/477] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Sep 19 16:00:05 2022] POST /identity/v3/users => generated 249 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 16:00:05.573309 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ae60da-23bb-4040-9ed7-1a849202d1ee None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:05.626210 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2576f6ea-bb50-44c8-bbe4-3a59795ff462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:05.626925 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576f6ea-bb50-44c8-bbe4-3a59795ff462 None admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:05.627214 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576f6ea-bb50-44c8-bbe4-3a59795ff462 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:05.627492 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576f6ea-bb50-44c8-bbe4-3a59795ff462 None admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:05.628186 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2576f6ea-bb50-44c8-bbe4-3a59795ff462 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:05.629594 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2576f6ea-bb50-44c8-bbe4-3a59795ff462 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:05.640122 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.sql.core [None req-2576f6ea-bb50-44c8-bbe4-3a59795ff462 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 19 16:00:05.640122 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 19 16:00:05.640122 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [parameters: {'id': 'b29bc4eec437478290df15295165f3a8', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 19 16:00:05.640122 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81618) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 19 16:00:05.659808 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-2576f6ea-bb50-44c8-bbe4-3a59795ff462 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 19 16:00:05.659808 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:05.659808 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 19 16:00:05.659808 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 19 16:00:05.659808 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 19 16:00:05.659808 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 19 16:00:05.659808 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 19 16:00:05.659808 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application result = self._query(query) Sep 19 16:00:05.659808 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 19 16:00:05.659808 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application conn.query(q) Sep 19 16:00:05.659808 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 19 16:00:05.659808 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 19 16:00:05.659808 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 19 16:00:05.659808 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application result.read() Sep 19 16:00:05.659808 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 19 16:00:05.660757 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 19 16:00:05.660757 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 19 16:00:05.660757 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 19 16:00:05.660757 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 19 16:00:05.660757 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 19 16:00:05.660757 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 19 16:00:05.660757 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 19 16:00:05.660757 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 19 16:00:05.660757 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:05.660757 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 19 16:00:05.660757 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:05.660757 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:05.660757 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 19 16:00:05.660757 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 19 16:00:05.660757 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 19 16:00:05.660757 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return ref.to_dict() Sep 19 16:00:05.660757 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 19 16:00:05.660757 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application next(self.gen) Sep 19 16:00:05.661660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 19 16:00:05.661660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application yield resource Sep 19 16:00:05.661660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 19 16:00:05.661660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application next(self.gen) Sep 19 16:00:05.661660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 19 16:00:05.661660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application self.session.rollback() Sep 19 16:00:05.661660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 19 16:00:05.661660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application self.force_reraise() Sep 19 16:00:05.661660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 19 16:00:05.661660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise self.value Sep 19 16:00:05.661660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 19 16:00:05.661660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 19 16:00:05.661660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 19 16:00:05.661660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application session.commit() Sep 19 16:00:05.661660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 19 16:00:05.661660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 19 16:00:05.661660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 19 16:00:05.663256 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application self._prepare_impl() Sep 19 16:00:05.663256 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 19 16:00:05.663256 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application self.session.flush() Sep 19 16:00:05.663256 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 19 16:00:05.663256 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application self._flush(objects) Sep 19 16:00:05.663256 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 19 16:00:05.663256 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 19 16:00:05.663256 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 19 16:00:05.663256 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application compat.raise_( Sep 19 16:00:05.663256 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 19 16:00:05.663256 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception Sep 19 16:00:05.663256 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 19 16:00:05.663256 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application flush_context.execute() Sep 19 16:00:05.663256 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 19 16:00:05.663256 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rec.execute(self) Sep 19 16:00:05.663256 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 19 16:00:05.663256 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 19 16:00:05.665267 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 19 16:00:05.665267 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 19 16:00:05.665267 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 19 16:00:05.665267 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 19 16:00:05.665267 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 19 16:00:05.665267 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 19 16:00:05.665267 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 19 16:00:05.665267 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 19 16:00:05.665267 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 19 16:00:05.665267 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 19 16:00:05.665267 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 19 16:00:05.665267 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 19 16:00:05.665267 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 19 16:00:05.665267 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 19 16:00:05.665267 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 19 16:00:05.665267 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception Sep 19 16:00:05.667564 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 19 16:00:05.667564 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 19 16:00:05.667564 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 19 16:00:05.667564 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 19 16:00:05.667564 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 19 16:00:05.667564 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application result = self._query(query) Sep 19 16:00:05.667564 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 19 16:00:05.667564 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application conn.query(q) Sep 19 16:00:05.667564 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 19 16:00:05.667564 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 19 16:00:05.667564 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 19 16:00:05.667564 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application result.read() Sep 19 16:00:05.667564 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 19 16:00:05.667564 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 19 16:00:05.667564 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 19 16:00:05.667564 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 19 16:00:05.669468 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 19 16:00:05.669468 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 19 16:00:05.669468 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 19 16:00:05.669468 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 19 16:00:05.669468 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 19 16:00:05.669468 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 19 16:00:05.669468 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [parameters: {'id': 'b29bc4eec437478290df15295165f3a8', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 19 16:00:05.669468 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 19 16:00:05.669468 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:05.669468 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 16:00:05.669468 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:05.669468 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:05.669468 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:05.669468 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:05.669468 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:05.669468 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:05.676012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:05.676012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:05.676012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:05.676012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:05.676012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:05.676012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:05.676012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 19 16:00:05.676012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 19 16:00:05.676012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:05.676012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:05.676012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 19 16:00:05.676012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 19 16:00:05.676012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 19 16:00:05.676012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 19 16:00:05.676012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 19 16:00:05.676012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:05.680545 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 233/478] 10.222.0.57 () {60 vars in 1131 bytes} [Mon Sep 19 16:00:05 2022] POST /identity/v3/roles => generated 144 bytes in 287 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:05.680700 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbf27d5-27d1-4f6f-b960-5760b57b110d None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:05.799159 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbf27d5-27d1-4f6f-b960-5760b57b110d None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:05.824358 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:05.835468 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749de674-44a8-4612-a944-bbfb69aa4b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:05.837202 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749de674-44a8-4612-a944-bbfb69aa4b77 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:05.837499 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749de674-44a8-4612-a944-bbfb69aa4b77 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:05.837764 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749de674-44a8-4612-a944-bbfb69aa4b77 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:05.849529 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749de674-44a8-4612-a944-bbfb69aa4b77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 16:00:05.854886 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-749de674-44a8-4612-a944-bbfb69aa4b77 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 16:00:05.854886 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:05.854886 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:05.854886 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:05.854886 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:05.854886 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:05.854886 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:05.854886 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:05.854886 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:05.854886 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:05.854886 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:05.854886 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:05.854886 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 16:00:05.854886 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 16:00:05.854886 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 16:00:05.854886 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 16:00:05.855835 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 16:00:05.855835 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 16:00:05.855835 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 16:00:05.855835 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 16:00:05.855835 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:05.855835 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:05.855835 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:05.855835 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:05.855835 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:05.855835 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:05.855835 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:05.855835 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:05.855835 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:05.855835 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:05.855835 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:05.855835 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:05.856608 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:05.856608 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:05.856608 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:05.856608 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:05.856608 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 16:00:05.856608 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 16:00:05.856608 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 16:00:05.856608 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 16:00:05.856608 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 16:00:05.856608 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 16:00:05.856608 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 16:00:05.856608 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:05.856608 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 246/479] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Sep 19 16:00:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:05.864764 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2576f6ea-bb50-44c8-bbe4-3a59795ff462 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:05.949876 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:05.977270 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2576f6ea-bb50-44c8-bbe4-3a59795ff462 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:06.013400 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ec5a8b-68ba-4fe4-966b-255822df34f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:06.014053 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ec5a8b-68ba-4fe4-966b-255822df34f6 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:06.014337 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ec5a8b-68ba-4fe4-966b-255822df34f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:06.014645 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ec5a8b-68ba-4fe4-966b-255822df34f6 None admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:06.015349 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ec5a8b-68ba-4fe4-966b-255822df34f6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:06.015656 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ec5a8b-68ba-4fe4-966b-255822df34f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:06.017126 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ec5a8b-68ba-4fe4-966b-255822df34f6 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:06.027386 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 234/480] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 19 16:00:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:00:06.834809 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d3da7-b5e0-4a29-8088-18e888559711 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:06.835316 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d3da7-b5e0-4a29-8088-18e888559711 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:06.835388 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d3da7-b5e0-4a29-8088-18e888559711 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:06.837136 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 247/481] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:00:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:00:06.843105 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8283fbfd-68e8-46d2-a1e2-42e228c5a9f3 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:06.843956 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8283fbfd-68e8-46d2-a1e2-42e228c5a9f3 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:06.844429 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8283fbfd-68e8-46d2-a1e2-42e228c5a9f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:06.870686 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-8283fbfd-68e8-46d2-a1e2-42e228c5a9f3 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:00:06.872219 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-8283fbfd-68e8-46d2-a1e2-42e228c5a9f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:00:06.975087 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 235/482] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:00:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 16:00:07.288228 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bbf91d-1d83-4ec3-89b1-238c6416d892 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:07.288558 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bbf91d-1d83-4ec3-89b1-238c6416d892 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:07.288795 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bbf91d-1d83-4ec3-89b1-238c6416d892 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:07.290435 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 248/483] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:00:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:00:07.296956 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9362730-f6db-4fd1-ada0-121683ff8c04 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:07.297243 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9362730-f6db-4fd1-ada0-121683ff8c04 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:07.297517 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9362730-f6db-4fd1-ada0-121683ff8c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:07.309199 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bbf91d-1d83-4ec3-89b1-238c6416d892 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:07.338685 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-a9362730-f6db-4fd1-ada0-121683ff8c04 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:00:07.339900 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-a9362730-f6db-4fd1-ada0-121683ff8c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:00:07.459416 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:07.479859 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bbf91d-1d83-4ec3-89b1-238c6416d892 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:07.481158 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 236/484] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:00:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 16:00:07.518671 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8c0f21-679d-48c2-a046-09fb8b009650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:07.519297 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c0f21-679d-48c2-a046-09fb8b009650 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:07.519631 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c0f21-679d-48c2-a046-09fb8b009650 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:07.519909 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c0f21-679d-48c2-a046-09fb8b009650 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:07.530992 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8c0f21-679d-48c2-a046-09fb8b009650 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 16:00:07.535763 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-fc8c0f21-679d-48c2-a046-09fb8b009650 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 16:00:07.535763 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:07.535763 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:07.535763 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:07.535763 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:07.535763 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:07.535763 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:07.535763 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:07.535763 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:07.535763 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:07.535763 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:07.535763 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:07.535763 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 16:00:07.535763 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 16:00:07.535763 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 16:00:07.535763 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 16:00:07.536627 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 16:00:07.536627 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 16:00:07.536627 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 16:00:07.536627 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 16:00:07.536627 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:07.536627 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:07.536627 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:07.536627 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:07.536627 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:07.536627 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:07.536627 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:07.536627 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:07.536627 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:07.536627 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:07.536627 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:07.536627 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:07.537390 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:07.537390 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:07.537390 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:07.537390 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:07.537390 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 16:00:07.537390 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 16:00:07.537390 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 16:00:07.537390 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 16:00:07.537390 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 16:00:07.537390 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 16:00:07.537390 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 16:00:07.537390 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:07.541985 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 249/485] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Sep 19 16:00:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:07.543302 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9362730-f6db-4fd1-ada0-121683ff8c04 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:07.643913 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9362730-f6db-4fd1-ada0-121683ff8c04 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:07.678134 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a75532-13a6-4588-96d9-f6df0d02cc2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:07.678768 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a75532-13a6-4588-96d9-f6df0d02cc2f None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:07.679039 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a75532-13a6-4588-96d9-f6df0d02cc2f None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:07.679309 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a75532-13a6-4588-96d9-f6df0d02cc2f None admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:07.680121 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a75532-13a6-4588-96d9-f6df0d02cc2f None admin] RBAC: Adding query filter params (name=service) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:07.680478 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a75532-13a6-4588-96d9-f6df0d02cc2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:07.681845 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a75532-13a6-4588-96d9-f6df0d02cc2f None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:07.690812 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 237/486] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Sep 19 16:00:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:00:07.696648 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8c0f21-679d-48c2-a046-09fb8b009650 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:07.758009 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:07.833752 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8c0f21-679d-48c2-a046-09fb8b009650 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:07.871439 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d8721f-17e0-434a-adc1-aa6cc5c99667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:07.872173 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8721f-17e0-434a-adc1-aa6cc5c99667 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:07.872400 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8721f-17e0-434a-adc1-aa6cc5c99667 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:07.873358 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8721f-17e0-434a-adc1-aa6cc5c99667 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:07.884037 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d8721f-17e0-434a-adc1-aa6cc5c99667 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:07.885606 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d8721f-17e0-434a-adc1-aa6cc5c99667 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:07.887930 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a75532-13a6-4588-96d9-f6df0d02cc2f None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:07.905524 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-66d8721f-17e0-434a-adc1-aa6cc5c99667 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:07.905524 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:07.905524 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:07.905524 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:07.905524 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:07.905524 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:07.905524 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:07.905524 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:07.905524 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:07.905524 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:07.905524 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:07.905524 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:07.905524 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:00:07.905524 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:00:07.905524 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:00:07.905524 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:00:07.906552 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:07.906552 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:07.906552 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:07.906552 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:07.906552 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:07.906552 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:07.906552 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:07.906552 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:07.906552 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:07.906552 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:07.906552 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:07.906552 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:07.906552 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:07.906552 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:07.906552 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:07.906552 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:07.906552 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:00:07.907385 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:00:07.907385 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:07.907385 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:07.911246 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 250/487] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:00:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:07.913899 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d8721f-17e0-434a-adc1-aa6cc5c99667 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:07.994772 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:08.035750 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:08.043320 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d8721f-17e0-434a-adc1-aa6cc5c99667 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:08.048036 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a75532-13a6-4588-96d9-f6df0d02cc2f None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:08.081097 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c517a2c-14f2-41ea-9bf9-ed438d6ecc07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:08.081800 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c517a2c-14f2-41ea-9bf9-ed438d6ecc07 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:08.082254 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c517a2c-14f2-41ea-9bf9-ed438d6ecc07 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:08.082627 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c517a2c-14f2-41ea-9bf9-ed438d6ecc07 None admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:08.083356 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c517a2c-14f2-41ea-9bf9-ed438d6ecc07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:08.083731 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c517a2c-14f2-41ea-9bf9-ed438d6ecc07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:08.085428 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c517a2c-14f2-41ea-9bf9-ed438d6ecc07 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:08.085540 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df16ea4e-e90d-497d-938a-f755332893c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:08.086028 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df16ea4e-e90d-497d-938a-f755332893c2 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:08.086306 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df16ea4e-e90d-497d-938a-f755332893c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:08.086605 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df16ea4e-e90d-497d-938a-f755332893c2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:08.098090 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df16ea4e-e90d-497d-938a-f755332893c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 16:00:08.098860 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 251/488] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:00:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:08.100240 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7291e571-1dd7-4902-9f35-7e50d6039519 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:08.100593 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7291e571-1dd7-4902-9f35-7e50d6039519 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:08.100945 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7291e571-1dd7-4902-9f35-7e50d6039519 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:08.103750 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-df16ea4e-e90d-497d-938a-f755332893c2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 16:00:08.103750 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:08.103750 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:08.103750 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:08.103750 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:08.103750 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:08.103750 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:08.103750 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:08.103750 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:08.103750 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:08.103750 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:08.103750 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:08.103750 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 16:00:08.103750 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 16:00:08.103750 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 16:00:08.103750 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 16:00:08.104700 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 16:00:08.104700 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 16:00:08.104700 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 16:00:08.104700 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 16:00:08.104700 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:08.104700 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:08.104700 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:08.104700 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:08.104700 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:08.104700 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:08.104700 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:08.104700 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:08.104700 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:08.104700 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:08.104700 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:08.104700 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:08.105515 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:08.105515 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:08.105515 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:08.105515 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:08.105515 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 16:00:08.105515 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 16:00:08.105515 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 16:00:08.105515 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 16:00:08.105515 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 16:00:08.105515 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 16:00:08.105515 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 16:00:08.105515 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:08.105515 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 238/490] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Sep 19 16:00:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 16:00:08.107685 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 252/490] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:00:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:00:08.111320 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df16ea4e-e90d-497d-938a-f755332893c2 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:08.111526 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e33b74-fc7c-40f7-8bfa-9f1b38ab2beb None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:08.111526 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e33b74-fc7c-40f7-8bfa-9f1b38ab2beb None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:08.111526 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e33b74-fc7c-40f7-8bfa-9f1b38ab2beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:08.137298 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-b0e33b74-fc7c-40f7-8bfa-9f1b38ab2beb None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:00:08.138680 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-b0e33b74-fc7c-40f7-8bfa-9f1b38ab2beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:00:08.218733 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 253/491] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:00:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 16:00:08.219997 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e33b74-fc7c-40f7-8bfa-9f1b38ab2beb None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:08.227403 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df16ea4e-e90d-497d-938a-f755332893c2 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:08.257735 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:08.287548 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b8a73e-520c-43e8-a9d5-e72b91eb4184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:08.288174 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b8a73e-520c-43e8-a9d5-e72b91eb4184 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:08.291092 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b8a73e-520c-43e8-a9d5-e72b91eb4184 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:08.291641 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b8a73e-520c-43e8-a9d5-e72b91eb4184 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:08.309050 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b8a73e-520c-43e8-a9d5-e72b91eb4184 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:08.310598 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b8a73e-520c-43e8-a9d5-e72b91eb4184 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:08.334263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-b9b8a73e-520c-43e8-a9d5-e72b91eb4184 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 19 16:00:08.334263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:08.334263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:08.334263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:08.334263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:08.334263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:08.334263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:08.334263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:08.334263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:08.334263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:08.334263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:08.334263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:08.334263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 16:00:08.334263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 16:00:08.334263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 16:00:08.334263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 16:00:08.336550 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:08.336550 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:08.336550 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 16:00:08.336550 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:00:08.336550 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 16:00:08.336550 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:00:08.336550 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:08.336550 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:08.336550 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:08.336550 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:08.336550 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:08.336550 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:08.336550 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:08.336550 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:08.336550 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:08.336550 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:08.336550 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:08.336550 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:08.338638 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:08.338638 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:08.338638 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 16:00:08.338638 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 16:00:08.338638 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 16:00:08.338638 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 16:00:08.338638 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 16:00:08.338638 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 16:00:08.338638 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 19 16:00:08.338638 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:08.338638 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 239/492] 10.222.0.57 () {56 vars in 1093 bytes} [Mon Sep 19 16:00:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:08.347770 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f56999-f65e-47eb-b37b-e8dc29e5a90b None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:08.348046 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f56999-f65e-47eb-b37b-e8dc29e5a90b None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:08.348316 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f56999-f65e-47eb-b37b-e8dc29e5a90b None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:08.350884 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 240/493] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:00:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:00:08.350884 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f56999-f65e-47eb-b37b-e8dc29e5a90b None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:08.383076 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:08.407800 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e33b74-fc7c-40f7-8bfa-9f1b38ab2beb None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:08.449761 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021612d9-11ac-476f-a663-81e9bb50d285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:08.450887 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021612d9-11ac-476f-a663-81e9bb50d285 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:08.451399 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021612d9-11ac-476f-a663-81e9bb50d285 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:08.451869 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021612d9-11ac-476f-a663-81e9bb50d285 None admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:08.452735 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021612d9-11ac-476f-a663-81e9bb50d285 None admin] RBAC: Adding query filter params (name=service) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:08.453316 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021612d9-11ac-476f-a663-81e9bb50d285 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:08.454941 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021612d9-11ac-476f-a663-81e9bb50d285 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:08.464446 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 254/494] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Sep 19 16:00:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:08.466152 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86c5adf-66b8-4a8c-ae83-cd48ecd55e25 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:08.466669 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86c5adf-66b8-4a8c-ae83-cd48ecd55e25 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:08.467206 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86c5adf-66b8-4a8c-ae83-cd48ecd55e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:08.480853 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f56999-f65e-47eb-b37b-e8dc29e5a90b None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:08.488688 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-d86c5adf-66b8-4a8c-ae83-cd48ecd55e25 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:00:08.490287 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-d86c5adf-66b8-4a8c-ae83-cd48ecd55e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:00:08.528009 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1830768b-0f29-4d76-b9d1-6e2cfca6674c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:08.528618 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1830768b-0f29-4d76-b9d1-6e2cfca6674c None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:08.528930 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1830768b-0f29-4d76-b9d1-6e2cfca6674c None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:08.529197 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1830768b-0f29-4d76-b9d1-6e2cfca6674c None admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:08.529926 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1830768b-0f29-4d76-b9d1-6e2cfca6674c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:08.530228 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1830768b-0f29-4d76-b9d1-6e2cfca6674c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:08.531828 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1830768b-0f29-4d76-b9d1-6e2cfca6674c None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:08.568325 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 255/495] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:00:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 16:00:08.571316 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86c5adf-66b8-4a8c-ae83-cd48ecd55e25 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:08.582048 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 241/496] 10.222.0.57 () {56 vars in 1133 bytes} [Mon Sep 19 16:00:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:08.582048 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1830768b-0f29-4d76-b9d1-6e2cfca6674c None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:08.628201 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:08.669645 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:08.696655 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86c5adf-66b8-4a8c-ae83-cd48ecd55e25 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:08.711233 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1830768b-0f29-4d76-b9d1-6e2cfca6674c None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:08.735674 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2c820b-470a-4b3e-a212-9c404d9851a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:08.736357 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2c820b-470a-4b3e-a212-9c404d9851a7 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:08.736639 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2c820b-470a-4b3e-a212-9c404d9851a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:08.736912 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2c820b-470a-4b3e-a212-9c404d9851a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:08.748638 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2c820b-470a-4b3e-a212-9c404d9851a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:08.750264 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2c820b-470a-4b3e-a212-9c404d9851a7 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:08.756939 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d388c705-1b13-482c-92ba-4eaeb2d191eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:08.758498 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d388c705-1b13-482c-92ba-4eaeb2d191eb None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:08.759017 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d388c705-1b13-482c-92ba-4eaeb2d191eb None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:08.759496 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d388c705-1b13-482c-92ba-4eaeb2d191eb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:08.770428 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-1a2c820b-470a-4b3e-a212-9c404d9851a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:08.770428 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:08.770428 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:08.770428 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:08.770428 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:08.770428 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:08.770428 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:08.770428 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:08.770428 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:08.770428 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:08.770428 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:08.770428 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:08.770428 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:00:08.770428 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:00:08.770428 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:00:08.770428 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:00:08.771357 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:08.771357 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:08.771357 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:08.771357 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:08.771357 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:08.771357 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:08.771357 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:08.771357 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:08.771357 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:08.771357 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:08.771357 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:08.771357 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:08.771357 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:08.771357 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:08.771357 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:08.771357 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:08.771357 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:00:08.772180 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:00:08.772180 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:08.772180 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:08.774955 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d388c705-1b13-482c-92ba-4eaeb2d191eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 16:00:08.776308 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 256/497] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:00:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:08.777214 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2c820b-470a-4b3e-a212-9c404d9851a7 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:08.782419 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-d388c705-1b13-482c-92ba-4eaeb2d191eb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 16:00:08.782419 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:08.782419 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:08.782419 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:08.782419 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:08.782419 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:08.782419 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:08.782419 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:08.782419 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:08.782419 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:08.782419 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:08.782419 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:08.782419 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 16:00:08.782419 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 16:00:08.782419 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 16:00:08.782419 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 16:00:08.783325 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 16:00:08.783325 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 16:00:08.783325 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 16:00:08.783325 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 16:00:08.783325 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:08.783325 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:08.783325 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:08.783325 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:08.783325 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:08.783325 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:08.783325 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:08.783325 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:08.783325 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:08.783325 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:08.783325 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:08.783325 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:08.784098 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:08.784098 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:08.784098 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:08.784098 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:08.784098 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 16:00:08.784098 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 16:00:08.784098 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 16:00:08.784098 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 16:00:08.784098 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 16:00:08.784098 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 16:00:08.784098 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 16:00:08.784098 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:08.786858 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 242/498] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Sep 19 16:00:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:08.788058 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d388c705-1b13-482c-92ba-4eaeb2d191eb None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:08.910288 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:08.920447 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d388c705-1b13-482c-92ba-4eaeb2d191eb None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:08.921165 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2c820b-470a-4b3e-a212-9c404d9851a7 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:08.952077 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:08.961602 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc4064-d701-4ee3-a693-0ed2025c83f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:08.962006 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ce72eb-3673-4fde-a3e5-2caa6bfd1b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:08.962210 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc4064-d701-4ee3-a693-0ed2025c83f8 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:08.962523 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc4064-d701-4ee3-a693-0ed2025c83f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:08.962703 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce72eb-3673-4fde-a3e5-2caa6bfd1b44 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:08.962788 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc4064-d701-4ee3-a693-0ed2025c83f8 None admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:08.962937 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce72eb-3673-4fde-a3e5-2caa6bfd1b44 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:08.963209 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce72eb-3673-4fde-a3e5-2caa6bfd1b44 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:08.963437 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bc4064-d701-4ee3-a693-0ed2025c83f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:08.963750 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bc4064-d701-4ee3-a693-0ed2025c83f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:08.965124 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bc4064-d701-4ee3-a693-0ed2025c83f8 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:08.977714 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 243/499] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:00:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:08.978590 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc4064-d701-4ee3-a693-0ed2025c83f8 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:09.033848 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ce72eb-3673-4fde-a3e5-2caa6bfd1b44 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:09.035334 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ce72eb-3673-4fde-a3e5-2caa6bfd1b44 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:09.080858 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc4064-d701-4ee3-a693-0ed2025c83f8 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:09.115357 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f462ad2-0244-45a7-9a92-ab33adfadf0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:09.115968 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f462ad2-0244-45a7-9a92-ab33adfadf0d None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:09.116247 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f462ad2-0244-45a7-9a92-ab33adfadf0d None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:09.116518 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f462ad2-0244-45a7-9a92-ab33adfadf0d None admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:09.117172 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f462ad2-0244-45a7-9a92-ab33adfadf0d None admin] RBAC: Adding query filter params (name=service) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:09.117477 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f462ad2-0244-45a7-9a92-ab33adfadf0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:09.118889 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f462ad2-0244-45a7-9a92-ab33adfadf0d None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:09.126914 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 244/500] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Sep 19 16:00:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:09.127731 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f462ad2-0244-45a7-9a92-ab33adfadf0d None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:09.142525 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 257/501] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 19 16:00:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 16:00:09.142525 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d31c1-4af9-4833-9cf3-68742a22afd7 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:09.142525 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d31c1-4af9-4833-9cf3-68742a22afd7 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:09.142525 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d31c1-4af9-4833-9cf3-68742a22afd7 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:09.142525 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 258/502] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:00:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:00:09.142525 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980d31c1-4af9-4833-9cf3-68742a22afd7 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:09.152409 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:09.170288 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:09.233591 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f462ad2-0244-45a7-9a92-ab33adfadf0d None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:09.244223 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980d31c1-4af9-4833-9cf3-68742a22afd7 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:09.267144 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698d3d02-b004-4664-8efe-0818c57a2272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:09.267768 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698d3d02-b004-4664-8efe-0818c57a2272 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:09.268043 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698d3d02-b004-4664-8efe-0818c57a2272 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:09.268319 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698d3d02-b004-4664-8efe-0818c57a2272 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:09.277558 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64443689-9b95-4291-b204-2db38dd32f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:09.278224 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64443689-9b95-4291-b204-2db38dd32f05 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:09.278326 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698d3d02-b004-4664-8efe-0818c57a2272 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 16:00:09.278585 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64443689-9b95-4291-b204-2db38dd32f05 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:09.278894 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64443689-9b95-4291-b204-2db38dd32f05 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:09.284660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-698d3d02-b004-4664-8efe-0818c57a2272 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 16:00:09.284660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:09.284660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:09.284660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:09.284660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:09.284660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:09.284660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:09.284660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:09.284660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:09.284660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:09.284660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:09.284660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:09.284660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 16:00:09.284660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 16:00:09.284660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 16:00:09.284660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 16:00:09.285564 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 16:00:09.285564 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 16:00:09.285564 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 16:00:09.285564 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 16:00:09.285564 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:09.285564 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:09.285564 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:09.285564 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:09.285564 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:09.285564 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:09.285564 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:09.285564 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:09.285564 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:09.285564 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:09.285564 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:09.285564 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:09.286365 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:09.286365 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:09.286365 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:09.286365 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:09.286365 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 16:00:09.286365 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 16:00:09.286365 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 16:00:09.286365 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 16:00:09.286365 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 16:00:09.286365 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 16:00:09.286365 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 16:00:09.286365 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:09.287057 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64443689-9b95-4291-b204-2db38dd32f05 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:09.287125 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 245/503] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Sep 19 16:00:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:09.287125 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698d3d02-b004-4664-8efe-0818c57a2272 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:09.287934 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64443689-9b95-4291-b204-2db38dd32f05 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:09.303866 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-64443689-9b95-4291-b204-2db38dd32f05 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 19 16:00:09.303866 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:09.303866 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:09.303866 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:09.303866 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:09.303866 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:09.303866 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:09.303866 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:09.303866 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:09.303866 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:09.303866 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:09.303866 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:09.303866 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 16:00:09.303866 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 16:00:09.303866 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 16:00:09.303866 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 16:00:09.304983 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:09.304983 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:09.304983 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 16:00:09.304983 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:00:09.304983 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 16:00:09.304983 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:00:09.304983 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:09.304983 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:09.304983 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:09.304983 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:09.304983 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:09.304983 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:09.304983 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:09.304983 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:09.304983 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:09.304983 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:09.304983 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:09.304983 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:09.305888 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:09.305888 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:09.305888 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 16:00:09.305888 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 16:00:09.305888 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 16:00:09.305888 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 16:00:09.305888 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 16:00:09.305888 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 16:00:09.305888 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 19 16:00:09.305888 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:09.305888 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 259/504] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 16:00:09 2022] GET /identity/v3/users/glance => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:09.305888 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64443689-9b95-4291-b204-2db38dd32f05 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:09.371699 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:09.391320 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:09.393853 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698d3d02-b004-4664-8efe-0818c57a2272 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:09.403744 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64443689-9b95-4291-b204-2db38dd32f05 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:09.427700 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33705e6c-9ea1-4020-9408-2bdf65bfe485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:09.428303 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33705e6c-9ea1-4020-9408-2bdf65bfe485 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:09.428590 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33705e6c-9ea1-4020-9408-2bdf65bfe485 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:09.428868 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33705e6c-9ea1-4020-9408-2bdf65bfe485 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:09.438085 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3422b1c2-19ef-4295-9c07-343ceab4cb27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:09.438836 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3422b1c2-19ef-4295-9c07-343ceab4cb27 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:09.439182 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3422b1c2-19ef-4295-9c07-343ceab4cb27 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:09.439499 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3422b1c2-19ef-4295-9c07-343ceab4cb27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:09.440484 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33705e6c-9ea1-4020-9408-2bdf65bfe485 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:09.441946 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33705e6c-9ea1-4020-9408-2bdf65bfe485 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:09.452237 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3422b1c2-19ef-4295-9c07-343ceab4cb27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:09.453699 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3422b1c2-19ef-4295-9c07-343ceab4cb27 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:09.457818 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-33705e6c-9ea1-4020-9408-2bdf65bfe485 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:09.457818 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:09.457818 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:09.457818 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:09.457818 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:09.457818 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:09.457818 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:09.457818 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:09.457818 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:09.457818 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:09.457818 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:09.457818 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:09.457818 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:00:09.457818 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:00:09.457818 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:00:09.457818 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:00:09.458770 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:09.458770 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:09.458770 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:09.458770 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:09.458770 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:09.458770 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:09.458770 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:09.458770 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:09.458770 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:09.458770 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:09.458770 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:09.458770 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:09.458770 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:09.458770 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:09.458770 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:09.458770 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:09.458770 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:00:09.459611 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:00:09.459611 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:09.459611 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:09.459611 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 246/505] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:00:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:09.460094 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1d45bc-1c18-41d6-b073-7992e7cf9481 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:09.460363 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1d45bc-1c18-41d6-b073-7992e7cf9481 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:09.460629 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1d45bc-1c18-41d6-b073-7992e7cf9481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:09.471978 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-3422b1c2-19ef-4295-9c07-343ceab4cb27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:09.471978 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:09.471978 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:09.471978 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:09.471978 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:09.471978 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:09.471978 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:09.471978 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:09.471978 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:09.471978 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:09.471978 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:09.471978 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:09.471978 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:00:09.471978 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:00:09.471978 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:00:09.471978 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:00:09.473028 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:09.473028 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:09.473028 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:09.473028 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:09.473028 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:09.473028 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:09.473028 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:09.473028 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:09.473028 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:09.473028 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:09.473028 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:09.473028 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:09.473028 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:09.473028 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:09.473028 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:09.473028 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:09.473028 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:00:09.473877 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:00:09.473877 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:09.473877 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:09.473877 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 260/506] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:00:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:09.473877 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3422b1c2-19ef-4295-9c07-343ceab4cb27 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:09.483233 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-1b1d45bc-1c18-41d6-b073-7992e7cf9481 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:00:09.484446 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-1b1d45bc-1c18-41d6-b073-7992e7cf9481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:00:09.551422 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 247/507] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:00:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 16:00:09.552281 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1d45bc-1c18-41d6-b073-7992e7cf9481 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:09.593106 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3422b1c2-19ef-4295-9c07-343ceab4cb27 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:09.630866 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:09.633234 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6859267-7cb1-4685-8d72-03d37d339013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:09.634193 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6859267-7cb1-4685-8d72-03d37d339013 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:09.634515 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6859267-7cb1-4685-8d72-03d37d339013 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:09.634848 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6859267-7cb1-4685-8d72-03d37d339013 None admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:09.635681 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6859267-7cb1-4685-8d72-03d37d339013 None admin] RBAC: Adding query filter params (name=service) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:09.635989 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6859267-7cb1-4685-8d72-03d37d339013 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:09.637369 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6859267-7cb1-4685-8d72-03d37d339013 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:09.650946 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 261/508] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Sep 19 16:00:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:09.650946 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6859267-7cb1-4685-8d72-03d37d339013 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:09.654412 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:09.680310 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1d45bc-1c18-41d6-b073-7992e7cf9481 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:09.717009 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb36b0a-8783-4fa6-8ea6-d803248b7df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:09.717938 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb36b0a-8783-4fa6-8ea6-d803248b7df5 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:09.718412 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb36b0a-8783-4fa6-8ea6-d803248b7df5 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:09.718712 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb36b0a-8783-4fa6-8ea6-d803248b7df5 None admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:09.719496 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb36b0a-8783-4fa6-8ea6-d803248b7df5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:09.719807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb36b0a-8783-4fa6-8ea6-d803248b7df5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:09.721247 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb36b0a-8783-4fa6-8ea6-d803248b7df5 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:09.739392 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 248/509] 10.222.0.57 () {56 vars in 1137 bytes} [Mon Sep 19 16:00:09 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:09.744650 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb36b0a-8783-4fa6-8ea6-d803248b7df5 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:09.760233 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6859267-7cb1-4685-8d72-03d37d339013 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:09.800771 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdebcb8-5a33-4c90-844f-4bb3512ae00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:09.801737 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdebcb8-5a33-4c90-844f-4bb3512ae00e None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:09.802241 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdebcb8-5a33-4c90-844f-4bb3512ae00e None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:09.802543 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdebcb8-5a33-4c90-844f-4bb3512ae00e None admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:09.803192 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdebcb8-5a33-4c90-844f-4bb3512ae00e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:09.803499 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdebcb8-5a33-4c90-844f-4bb3512ae00e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:09.804878 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdebcb8-5a33-4c90-844f-4bb3512ae00e None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:09.817376 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 262/510] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:00:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:09.818676 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdebcb8-5a33-4c90-844f-4bb3512ae00e None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:09.855649 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb36b0a-8783-4fa6-8ea6-d803248b7df5 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:09.873628 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:09.879763 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:09.891135 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b35667-ec9f-4373-867f-2f30f117e337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:09.891731 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b35667-ec9f-4373-867f-2f30f117e337 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:09.892005 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b35667-ec9f-4373-867f-2f30f117e337 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:09.892274 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b35667-ec9f-4373-867f-2f30f117e337 None admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:09.892906 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b35667-ec9f-4373-867f-2f30f117e337 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:09.893204 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b35667-ec9f-4373-867f-2f30f117e337 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:09.894595 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b35667-ec9f-4373-867f-2f30f117e337 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:09.910945 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 249/511] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:00:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:09.910945 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b35667-ec9f-4373-867f-2f30f117e337 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:09.922246 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdebcb8-5a33-4c90-844f-4bb3512ae00e None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:09.958575 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7416232-3b91-470c-b3b7-36ea3f8641e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:09.959209 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7416232-3b91-470c-b3b7-36ea3f8641e1 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:09.959493 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7416232-3b91-470c-b3b7-36ea3f8641e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:09.959770 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7416232-3b91-470c-b3b7-36ea3f8641e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:09.970649 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7416232-3b91-470c-b3b7-36ea3f8641e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:09.972097 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7416232-3b91-470c-b3b7-36ea3f8641e1 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:09.986610 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-c7416232-3b91-470c-b3b7-36ea3f8641e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:09.986610 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:09.986610 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:09.986610 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:09.986610 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:09.986610 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:09.986610 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:09.986610 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:09.986610 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:09.986610 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:09.986610 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:09.986610 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:09.986610 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:00:09.986610 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:00:09.986610 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:00:09.986610 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:00:09.987727 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:09.987727 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:09.987727 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:09.987727 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:09.987727 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:09.987727 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:09.987727 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:09.987727 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:09.987727 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:09.987727 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:09.987727 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:09.987727 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:09.987727 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:09.987727 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:09.987727 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:09.987727 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:09.987727 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:00:09.988579 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:00:09.988579 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:09.988579 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:09.988579 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 263/512] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:00:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:09.988841 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7416232-3b91-470c-b3b7-36ea3f8641e1 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:10.015488 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b35667-ec9f-4373-867f-2f30f117e337 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:10.049842 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe72db8-2c3c-4bc8-83b4-ac3c62049410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:10.050489 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe72db8-2c3c-4bc8-83b4-ac3c62049410 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:10.050766 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe72db8-2c3c-4bc8-83b4-ac3c62049410 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:10.051040 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe72db8-2c3c-4bc8-83b4-ac3c62049410 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:10.090814 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7416232-3b91-470c-b3b7-36ea3f8641e1 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:10.096315 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:10.105904 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:10.118080 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe72db8-2c3c-4bc8-83b4-ac3c62049410 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:10.119570 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe72db8-2c3c-4bc8-83b4-ac3c62049410 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:10.126616 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced1a750-387a-449d-873f-5e1f71c671db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:10.127250 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced1a750-387a-449d-873f-5e1f71c671db None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:10.127529 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced1a750-387a-449d-873f-5e1f71c671db None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:10.127803 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced1a750-387a-449d-873f-5e1f71c671db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:10.138935 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced1a750-387a-449d-873f-5e1f71c671db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:10.140550 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced1a750-387a-449d-873f-5e1f71c671db None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:10.155551 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-ced1a750-387a-449d-873f-5e1f71c671db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:10.155551 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:10.155551 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:10.155551 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:10.155551 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:10.155551 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:10.155551 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:10.155551 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:10.155551 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:10.155551 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:10.155551 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:10.155551 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:10.155551 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:00:10.155551 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:00:10.155551 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:00:10.155551 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:00:10.156463 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:10.156463 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:10.156463 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:10.156463 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:10.156463 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:10.156463 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:10.156463 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:10.156463 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:10.156463 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:10.156463 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:10.156463 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:10.156463 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:10.156463 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:10.156463 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:10.156463 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:10.156463 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:10.156463 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:00:10.157301 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:00:10.157301 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:10.157301 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:10.157301 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 264/513] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:00:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:10.157521 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced1a750-387a-449d-873f-5e1f71c671db None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:10.220388 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 250/514] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 19 16:00:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 16:00:10.221154 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe72db8-2c3c-4bc8-83b4-ac3c62049410 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:10.265291 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced1a750-387a-449d-873f-5e1f71c671db None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:10.298532 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:10.301128 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d3632c-1a38-4f5e-87c7-6b5d585610d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:10.301790 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d3632c-1a38-4f5e-87c7-6b5d585610d1 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:10.302075 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d3632c-1a38-4f5e-87c7-6b5d585610d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:10.302347 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d3632c-1a38-4f5e-87c7-6b5d585610d1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:10.310657 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d3632c-1a38-4f5e-87c7-6b5d585610d1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:10.312113 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d3632c-1a38-4f5e-87c7-6b5d585610d1 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:10.321511 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe72db8-2c3c-4bc8-83b4-ac3c62049410 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:10.323265 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-37d3632c-1a38-4f5e-87c7-6b5d585610d1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 19 16:00:10.323265 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:10.323265 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:10.323265 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:10.323265 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:10.323265 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:10.323265 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:10.323265 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:10.323265 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:10.323265 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:10.323265 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:10.323265 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:10.323265 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 16:00:10.323265 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 16:00:10.323265 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 16:00:10.323265 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 16:00:10.324291 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:10.324291 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:10.324291 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 16:00:10.324291 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:00:10.324291 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 16:00:10.324291 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:00:10.324291 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:10.324291 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:10.324291 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:10.324291 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:10.324291 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:10.324291 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:10.324291 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:10.324291 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:10.324291 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:10.324291 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:10.324291 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:10.324291 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:10.325386 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:10.325386 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:10.325386 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 16:00:10.325386 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 16:00:10.325386 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 16:00:10.325386 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 16:00:10.325386 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 16:00:10.325386 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 16:00:10.325386 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 19 16:00:10.325386 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:10.325386 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 265/515] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 16:00:10 2022] GET /identity/v3/users/cinder => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:10.325386 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d3632c-1a38-4f5e-87c7-6b5d585610d1 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:10.353843 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:10.359051 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e7ebab-cf0e-4e80-bc80-d59b99369c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:10.359687 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e7ebab-cf0e-4e80-bc80-d59b99369c4b None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:10.359964 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e7ebab-cf0e-4e80-bc80-d59b99369c4b None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:10.360237 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e7ebab-cf0e-4e80-bc80-d59b99369c4b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:10.367808 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e7ebab-cf0e-4e80-bc80-d59b99369c4b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:10.369224 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e7ebab-cf0e-4e80-bc80-d59b99369c4b None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:10.381368 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-41e7ebab-cf0e-4e80-bc80-d59b99369c4b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:00:10.381368 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:10.381368 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:10.381368 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:10.381368 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:10.381368 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:10.381368 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:10.381368 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:10.381368 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:10.381368 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:10.381368 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:10.381368 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:10.381368 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 16:00:10.381368 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 16:00:10.381368 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 16:00:10.382427 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 16:00:10.382427 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:10.382427 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:10.382427 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:10.382427 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:10.382427 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:10.382427 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:10.382427 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:10.382427 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:10.382427 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:10.382427 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:10.382427 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:10.382427 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:10.382427 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:10.382427 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:10.382427 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:10.382427 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:10.383305 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 16:00:10.383305 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 16:00:10.383305 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 16:00:10.383305 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 16:00:10.383305 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 16:00:10.383305 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 16:00:10.383305 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:00:10.383305 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:10.383305 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 251/516] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 19 16:00:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:10.383768 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e7ebab-cf0e-4e80-bc80-d59b99369c4b None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:10.436360 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d3632c-1a38-4f5e-87c7-6b5d585610d1 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:10.470105 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c1b0c3-2339-4644-bfbf-edebad7765e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:10.470792 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c1b0c3-2339-4644-bfbf-edebad7765e2 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:10.471015 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c1b0c3-2339-4644-bfbf-edebad7765e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:10.471288 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c1b0c3-2339-4644-bfbf-edebad7765e2 None admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:10.471929 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c1b0c3-2339-4644-bfbf-edebad7765e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:10.472238 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c1b0c3-2339-4644-bfbf-edebad7765e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:10.473600 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c1b0c3-2339-4644-bfbf-edebad7765e2 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:10.486132 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e7ebab-cf0e-4e80-bc80-d59b99369c4b None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:10.486293 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 266/517] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:00:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:10.486609 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c1b0c3-2339-4644-bfbf-edebad7765e2 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:10.523758 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14332a83-67a9-4e23-aa6c-6a9c38b6622b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:10.524361 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14332a83-67a9-4e23-aa6c-6a9c38b6622b None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:10.524634 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14332a83-67a9-4e23-aa6c-6a9c38b6622b None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:10.524900 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14332a83-67a9-4e23-aa6c-6a9c38b6622b None admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:10.525531 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14332a83-67a9-4e23-aa6c-6a9c38b6622b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:10.525832 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14332a83-67a9-4e23-aa6c-6a9c38b6622b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:10.527232 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14332a83-67a9-4e23-aa6c-6a9c38b6622b None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:10.534164 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:10.539981 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 252/518] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:00:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:10.540798 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14332a83-67a9-4e23-aa6c-6a9c38b6622b None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:10.572875 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:10.589484 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c1b0c3-2339-4644-bfbf-edebad7765e2 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:10.623054 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723c6fbe-c1fb-411f-9444-85f27a43a1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:10.623679 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723c6fbe-c1fb-411f-9444-85f27a43a1c9 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:10.623958 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723c6fbe-c1fb-411f-9444-85f27a43a1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:10.624225 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723c6fbe-c1fb-411f-9444-85f27a43a1c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:10.634329 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723c6fbe-c1fb-411f-9444-85f27a43a1c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:10.635802 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723c6fbe-c1fb-411f-9444-85f27a43a1c9 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:10.641039 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14332a83-67a9-4e23-aa6c-6a9c38b6622b None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:10.650888 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-723c6fbe-c1fb-411f-9444-85f27a43a1c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:10.650888 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:10.650888 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:10.650888 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:10.650888 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:10.650888 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:10.650888 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:10.650888 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:10.650888 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:10.650888 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:10.650888 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:10.650888 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:10.650888 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:00:10.650888 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:00:10.650888 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:00:10.650888 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:00:10.651885 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:10.651885 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:10.651885 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:10.651885 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:10.651885 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:10.651885 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:10.651885 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:10.651885 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:10.651885 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:10.651885 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:10.651885 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:10.651885 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:10.651885 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:10.651885 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:10.651885 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:10.651885 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:10.651885 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:00:10.652705 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:00:10.652705 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:10.652705 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:10.652705 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 267/519] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:00:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:10.652927 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723c6fbe-c1fb-411f-9444-85f27a43a1c9 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:10.682123 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff6ee65-f6e7-4226-b570-a2ab6fe75e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:10.682826 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff6ee65-f6e7-4226-b570-a2ab6fe75e4d None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:10.683109 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff6ee65-f6e7-4226-b570-a2ab6fe75e4d None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:10.683382 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff6ee65-f6e7-4226-b570-a2ab6fe75e4d None admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:10.684103 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff6ee65-f6e7-4226-b570-a2ab6fe75e4d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:10.684405 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff6ee65-f6e7-4226-b570-a2ab6fe75e4d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:10.685869 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff6ee65-f6e7-4226-b570-a2ab6fe75e4d None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:10.704052 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 253/520] 10.222.0.57 () {56 vars in 1137 bytes} [Mon Sep 19 16:00:10 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:10.704869 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff6ee65-f6e7-4226-b570-a2ab6fe75e4d None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:10.755939 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723c6fbe-c1fb-411f-9444-85f27a43a1c9 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:10.765234 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:10.791820 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3287af1-b118-467b-898e-0f7ab1c57ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:10.792430 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3287af1-b118-467b-898e-0f7ab1c57ef3 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:10.792710 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3287af1-b118-467b-898e-0f7ab1c57ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:10.792983 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3287af1-b118-467b-898e-0f7ab1c57ef3 None admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:10.793678 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3287af1-b118-467b-898e-0f7ab1c57ef3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:10.793984 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3287af1-b118-467b-898e-0f7ab1c57ef3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:10.795518 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3287af1-b118-467b-898e-0f7ab1c57ef3 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:10.806037 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:10.810284 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff6ee65-f6e7-4226-b570-a2ab6fe75e4d None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:10.814414 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 268/521] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 19 16:00:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:10.815231 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3287af1-b118-467b-898e-0f7ab1c57ef3 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:10.845851 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96f2ce6-15f9-44a6-96ed-1933806af69e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:10.846506 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96f2ce6-15f9-44a6-96ed-1933806af69e None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:10.846798 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96f2ce6-15f9-44a6-96ed-1933806af69e None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:10.847069 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96f2ce6-15f9-44a6-96ed-1933806af69e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:10.854200 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96f2ce6-15f9-44a6-96ed-1933806af69e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:10.855660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96f2ce6-15f9-44a6-96ed-1933806af69e None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:10.867141 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-b96f2ce6-15f9-44a6-96ed-1933806af69e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 19 16:00:10.867141 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:10.867141 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:10.867141 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:10.867141 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:10.867141 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:10.867141 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:10.867141 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:10.867141 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:10.867141 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:10.867141 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:10.867141 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:10.867141 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 16:00:10.867141 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 16:00:10.867141 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 16:00:10.867141 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 16:00:10.868320 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:10.868320 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:10.868320 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 16:00:10.868320 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:00:10.868320 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 16:00:10.868320 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:00:10.868320 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:10.868320 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:10.868320 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:10.868320 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:10.868320 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:10.868320 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:10.868320 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:10.868320 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:10.868320 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:10.868320 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:10.868320 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:10.868320 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:10.869191 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:10.869191 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:10.869191 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 16:00:10.869191 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 16:00:10.869191 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 16:00:10.869191 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 16:00:10.869191 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 16:00:10.869191 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 16:00:10.869191 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 19 16:00:10.869191 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:10.869191 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 254/522] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Sep 19 16:00:10 2022] GET /identity/v3/users/neutron => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:10.869191 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96f2ce6-15f9-44a6-96ed-1933806af69e None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:10.923691 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3287af1-b118-467b-898e-0f7ab1c57ef3 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:10.959337 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b03c94c-9009-4236-bc0e-7c36e0f94f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:10.959950 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b03c94c-9009-4236-bc0e-7c36e0f94f01 None admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:10.960228 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b03c94c-9009-4236-bc0e-7c36e0f94f01 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:10.960494 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b03c94c-9009-4236-bc0e-7c36e0f94f01 None admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:10.961146 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b03c94c-9009-4236-bc0e-7c36e0f94f01 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:10.962621 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b03c94c-9009-4236-bc0e-7c36e0f94f01 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:10.982545 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96f2ce6-15f9-44a6-96ed-1933806af69e None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:10.985235 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 269/523] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Sep 19 16:00:10 2022] POST /identity/v3/users => generated 247 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:00:10.986148 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b03c94c-9009-4236-bc0e-7c36e0f94f01 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:11.006767 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:11.031619 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ed3e62-f6df-44f4-8b96-4e95e7518a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:11.032184 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ed3e62-f6df-44f4-8b96-4e95e7518a2b None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:11.032569 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ed3e62-f6df-44f4-8b96-4e95e7518a2b None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:11.032907 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ed3e62-f6df-44f4-8b96-4e95e7518a2b None admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:11.033640 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ed3e62-f6df-44f4-8b96-4e95e7518a2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:11.034008 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ed3e62-f6df-44f4-8b96-4e95e7518a2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:11.035696 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ed3e62-f6df-44f4-8b96-4e95e7518a2b None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:11.052824 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 255/524] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:00:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:11.054006 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ed3e62-f6df-44f4-8b96-4e95e7518a2b None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:11.060449 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:11.120223 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b03c94c-9009-4236-bc0e-7c36e0f94f01 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:11.159779 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c46141-16e3-4c16-8e87-88b690fb89a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:11.160659 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c46141-16e3-4c16-8e87-88b690fb89a7 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:11.160931 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c46141-16e3-4c16-8e87-88b690fb89a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:11.161202 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c46141-16e3-4c16-8e87-88b690fb89a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:11.196151 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ed3e62-f6df-44f4-8b96-4e95e7518a2b None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:11.239158 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60be1a6-9ee2-40f7-8125-e373e9580e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:11.239786 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60be1a6-9ee2-40f7-8125-e373e9580e7b None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:11.240204 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60be1a6-9ee2-40f7-8125-e373e9580e7b None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:11.240480 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60be1a6-9ee2-40f7-8125-e373e9580e7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:11.241369 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60be1a6-9ee2-40f7-8125-e373e9580e7b None admin] RBAC: Adding query filter params (role.id=1e475e6a170a4dbabc4ce8348eeb30b9, scope.project.id=be03e5300b3343169f35c643c93d1e06, user.id=1ecd01cc72834b72b5ff2c4b47249737) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:11.241689 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60be1a6-9ee2-40f7-8125-e373e9580e7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:11.243220 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60be1a6-9ee2-40f7-8125-e373e9580e7b None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:11.244244 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c46141-16e3-4c16-8e87-88b690fb89a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:11.245716 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c46141-16e3-4c16-8e87-88b690fb89a7 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:11.254404 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 256/525] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 19 16:00:11 2022] GET /identity/v3/role_assignments?user.id=1ecd01cc72834b72b5ff2c4b47249737&scope.project.id=be03e5300b3343169f35c643c93d1e06&role.id=1e475e6a170a4dbabc4ce8348eeb30b9 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:11.254404 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60be1a6-9ee2-40f7-8125-e373e9580e7b None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:11.321620 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:11.346401 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:11.405645 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60be1a6-9ee2-40f7-8125-e373e9580e7b None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:11.424782 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 270/526] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 19 16:00:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 16:00:11.426150 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c46141-16e3-4c16-8e87-88b690fb89a7 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:11.452470 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c103786a-0376-4865-b717-a503b47b972f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:11.454064 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c103786a-0376-4865-b717-a503b47b972f None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:11.454064 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c103786a-0376-4865-b717-a503b47b972f None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:11.454421 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c103786a-0376-4865-b717-a503b47b972f None admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:11.455187 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c103786a-0376-4865-b717-a503b47b972f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:11.455501 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c103786a-0376-4865-b717-a503b47b972f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:11.457680 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c103786a-0376-4865-b717-a503b47b972f None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:11.478013 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 257/527] 10.222.0.57 () {56 vars in 1139 bytes} [Mon Sep 19 16:00:11 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:11.478894 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c103786a-0376-4865-b717-a503b47b972f None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:11.556146 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c46141-16e3-4c16-8e87-88b690fb89a7 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:11.592115 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7423e806-62b6-4ae6-a8b2-9e61e61c7437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:11.592662 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423e806-62b6-4ae6-a8b2-9e61e61c7437 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:11.592936 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423e806-62b6-4ae6-a8b2-9e61e61c7437 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:11.593204 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423e806-62b6-4ae6-a8b2-9e61e61c7437 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:11.600434 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7423e806-62b6-4ae6-a8b2-9e61e61c7437 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:11.601897 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7423e806-62b6-4ae6-a8b2-9e61e61c7437 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:11.605128 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:11.610875 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c103786a-0376-4865-b717-a503b47b972f None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:11.614991 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-7423e806-62b6-4ae6-a8b2-9e61e61c7437 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:00:11.614991 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:11.614991 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:11.614991 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:11.614991 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:11.614991 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:11.614991 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:11.614991 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:11.614991 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:11.614991 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:11.614991 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:11.614991 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:11.614991 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 16:00:11.614991 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 16:00:11.614991 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 16:00:11.615910 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 16:00:11.615910 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:11.615910 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:11.615910 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:11.615910 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:11.615910 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:11.615910 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:11.615910 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:11.615910 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:11.615910 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:11.615910 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:11.615910 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:11.615910 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:11.615910 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:11.615910 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:11.615910 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:11.615910 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:11.616745 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 16:00:11.616745 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 16:00:11.616745 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 16:00:11.616745 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 16:00:11.616745 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 16:00:11.616745 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 16:00:11.616745 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:00:11.616745 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:11.616745 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 271/528] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 19 16:00:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:11.617649 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7423e806-62b6-4ae6-a8b2-9e61e61c7437 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:11.634766 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:11.655690 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3f7d12-9a64-4d4f-a279-3b03dde0f450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:11.657311 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f7d12-9a64-4d4f-a279-3b03dde0f450 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:11.657700 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f7d12-9a64-4d4f-a279-3b03dde0f450 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:11.658090 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f7d12-9a64-4d4f-a279-3b03dde0f450 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:11.671762 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3f7d12-9a64-4d4f-a279-3b03dde0f450 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:11.673564 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3f7d12-9a64-4d4f-a279-3b03dde0f450 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:11.695347 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-fe3f7d12-9a64-4d4f-a279-3b03dde0f450 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:11.695347 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:11.695347 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:11.695347 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:11.695347 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:11.695347 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:11.695347 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:11.695347 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:11.695347 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:11.695347 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:11.695347 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:11.695347 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:11.695347 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:00:11.695347 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:00:11.695347 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:00:11.695347 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:00:11.696425 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:11.696425 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:11.696425 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:11.696425 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:11.696425 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:11.696425 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:11.696425 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:11.696425 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:11.696425 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:11.696425 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:11.696425 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:11.696425 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:11.696425 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:11.696425 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:11.696425 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:11.696425 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:11.696425 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:00:11.697398 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:00:11.697398 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:11.697398 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:11.699535 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 258/529] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:00:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:11.708607 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3f7d12-9a64-4d4f-a279-3b03dde0f450 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:11.732541 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7423e806-62b6-4ae6-a8b2-9e61e61c7437 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:11.771762 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0bc560-bfa5-4dde-9409-d0c7a774dfd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:11.772609 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0bc560-bfa5-4dde-9409-d0c7a774dfd1 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:11.773595 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0bc560-bfa5-4dde-9409-d0c7a774dfd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:11.774040 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0bc560-bfa5-4dde-9409-d0c7a774dfd1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:11.831186 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3f7d12-9a64-4d4f-a279-3b03dde0f450 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:11.869700 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3b79d4-fce9-454c-b713-c7de5ca2e034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:11.870387 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3b79d4-fce9-454c-b713-c7de5ca2e034 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:11.870643 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0bc560-bfa5-4dde-9409-d0c7a774dfd1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:11.870861 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3b79d4-fce9-454c-b713-c7de5ca2e034 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:11.871263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3b79d4-fce9-454c-b713-c7de5ca2e034 None admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:11.872095 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3b79d4-fce9-454c-b713-c7de5ca2e034 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:11.872585 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3b79d4-fce9-454c-b713-c7de5ca2e034 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:11.873380 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0bc560-bfa5-4dde-9409-d0c7a774dfd1 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:11.874191 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3b79d4-fce9-454c-b713-c7de5ca2e034 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:11.890760 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:11.897287 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 259/530] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 19 16:00:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:11.898135 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3b79d4-fce9-454c-b713-c7de5ca2e034 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:11.916611 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:12.012245 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 272/531] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 19 16:00:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 16:00:12.013795 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0bc560-bfa5-4dde-9409-d0c7a774dfd1 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:12.043352 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3b79d4-fce9-454c-b713-c7de5ca2e034 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:12.083557 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6850aec-f9fe-435b-9f9f-44786d02e866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:12.084351 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6850aec-f9fe-435b-9f9f-44786d02e866 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:12.084749 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6850aec-f9fe-435b-9f9f-44786d02e866 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:12.085411 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6850aec-f9fe-435b-9f9f-44786d02e866 None admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:12.085868 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6850aec-f9fe-435b-9f9f-44786d02e866 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:12.086272 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6850aec-f9fe-435b-9f9f-44786d02e866 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:12.087760 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6850aec-f9fe-435b-9f9f-44786d02e866 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:12.102014 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 260/532] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:00:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:12.102862 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6850aec-f9fe-435b-9f9f-44786d02e866 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:12.150112 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0bc560-bfa5-4dde-9409-d0c7a774dfd1 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:12.169961 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:12.189597 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:12.199230 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08f7086-4c23-4293-bf84-8e596cabd8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:12.202579 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08f7086-4c23-4293-bf84-8e596cabd8b5 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:12.202854 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08f7086-4c23-4293-bf84-8e596cabd8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:12.203193 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08f7086-4c23-4293-bf84-8e596cabd8b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:12.203958 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08f7086-4c23-4293-bf84-8e596cabd8b5 None admin] RBAC: Adding query filter params (role.id=1e475e6a170a4dbabc4ce8348eeb30b9, scope.project.id=be03e5300b3343169f35c643c93d1e06, user.id=5632ade0cc6f4ae28f72160de68486cb) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:12.204275 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08f7086-4c23-4293-bf84-8e596cabd8b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:12.205793 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08f7086-4c23-4293-bf84-8e596cabd8b5 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:12.212578 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 273/533] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 19 16:00:12 2022] GET /identity/v3/role_assignments?user.id=5632ade0cc6f4ae28f72160de68486cb&scope.project.id=be03e5300b3343169f35c643c93d1e06&role.id=1e475e6a170a4dbabc4ce8348eeb30b9 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:12.213467 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08f7086-4c23-4293-bf84-8e596cabd8b5 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:12.239671 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6850aec-f9fe-435b-9f9f-44786d02e866 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:12.285605 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c20a6c0-66f5-4470-9ef9-111da003ada1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:12.286797 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c20a6c0-66f5-4470-9ef9-111da003ada1 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:12.287278 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c20a6c0-66f5-4470-9ef9-111da003ada1 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:12.287768 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c20a6c0-66f5-4470-9ef9-111da003ada1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:12.301656 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c20a6c0-66f5-4470-9ef9-111da003ada1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:12.303640 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c20a6c0-66f5-4470-9ef9-111da003ada1 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:12.322436 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-5c20a6c0-66f5-4470-9ef9-111da003ada1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:12.322436 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:12.322436 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:12.322436 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:12.322436 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:12.322436 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:12.322436 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:12.322436 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:12.322436 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:12.322436 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:12.322436 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:12.322436 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:12.322436 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:00:12.322436 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:00:12.322436 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:00:12.322436 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:00:12.323394 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:12.323394 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:12.323394 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:12.323394 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:12.323394 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:12.323394 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:12.323394 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:12.323394 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:12.323394 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:12.323394 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:12.323394 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:12.323394 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:12.323394 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:12.323394 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:12.323394 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:12.323394 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:12.323394 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:00:12.324216 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:00:12.324216 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:12.324216 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:12.331842 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 261/534] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:00:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:12.334336 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c20a6c0-66f5-4470-9ef9-111da003ada1 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:12.336395 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08f7086-4c23-4293-bf84-8e596cabd8b5 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:12.375093 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4cee53-953a-410e-908b-3d85ef391f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:12.375738 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4cee53-953a-410e-908b-3d85ef391f13 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:12.376021 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4cee53-953a-410e-908b-3d85ef391f13 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:12.376294 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4cee53-953a-410e-908b-3d85ef391f13 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:12.383753 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4cee53-953a-410e-908b-3d85ef391f13 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:12.385210 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4cee53-953a-410e-908b-3d85ef391f13 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:12.403099 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-0f4cee53-953a-410e-908b-3d85ef391f13 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:00:12.403099 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:12.403099 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:12.403099 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:12.403099 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:12.403099 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:12.403099 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:12.403099 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:12.403099 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:12.403099 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:12.403099 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:12.403099 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:12.403099 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 16:00:12.403099 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 16:00:12.403099 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 16:00:12.404031 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 16:00:12.404031 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:12.404031 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:12.404031 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:12.404031 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:12.404031 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:12.404031 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:12.404031 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:12.404031 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:12.404031 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:12.404031 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:12.404031 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:12.404031 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:12.404031 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:12.404031 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:12.404031 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:12.404031 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:12.405476 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 16:00:12.405476 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 16:00:12.405476 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 16:00:12.405476 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 16:00:12.405476 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 16:00:12.405476 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 16:00:12.405476 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:00:12.405476 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:12.405476 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 274/535] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 19 16:00:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:12.411500 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4cee53-953a-410e-908b-3d85ef391f13 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:12.494733 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:12.515080 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c20a6c0-66f5-4470-9ef9-111da003ada1 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:12.567243 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:12.594061 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d017d7-7868-4cf0-8114-5fa066a8ad4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:12.594707 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d017d7-7868-4cf0-8114-5fa066a8ad4f None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:12.594985 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d017d7-7868-4cf0-8114-5fa066a8ad4f None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:12.595256 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d017d7-7868-4cf0-8114-5fa066a8ad4f None admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:12.595993 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d017d7-7868-4cf0-8114-5fa066a8ad4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:12.596349 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d017d7-7868-4cf0-8114-5fa066a8ad4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:12.597903 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d017d7-7868-4cf0-8114-5fa066a8ad4f None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:12.613542 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4cee53-953a-410e-908b-3d85ef391f13 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:12.626394 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 262/536] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:00:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:12.634184 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d017d7-7868-4cf0-8114-5fa066a8ad4f None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:12.698227 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b96f36-db4a-4fd2-b99f-dd2c7f7305d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:12.699101 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b96f36-db4a-4fd2-b99f-dd2c7f7305d0 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:12.699525 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b96f36-db4a-4fd2-b99f-dd2c7f7305d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:12.699939 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b96f36-db4a-4fd2-b99f-dd2c7f7305d0 None admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:12.701372 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b96f36-db4a-4fd2-b99f-dd2c7f7305d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:12.701823 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b96f36-db4a-4fd2-b99f-dd2c7f7305d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:12.707579 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b96f36-db4a-4fd2-b99f-dd2c7f7305d0 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:12.728837 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 275/537] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 19 16:00:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:12.734882 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b96f36-db4a-4fd2-b99f-dd2c7f7305d0 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:12.742165 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d017d7-7868-4cf0-8114-5fa066a8ad4f None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:12.822703 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa7176d-0ca8-4b5c-b936-094591446494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:12.822703 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa7176d-0ca8-4b5c-b936-094591446494 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:12.822703 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa7176d-0ca8-4b5c-b936-094591446494 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:12.822703 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa7176d-0ca8-4b5c-b936-094591446494 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:12.829068 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa7176d-0ca8-4b5c-b936-094591446494 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:12.836363 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa7176d-0ca8-4b5c-b936-094591446494 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:12.854241 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-4fa7176d-0ca8-4b5c-b936-094591446494 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:00:12.854241 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:12.854241 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:12.854241 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:12.854241 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:12.854241 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:12.854241 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:12.854241 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:12.854241 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:12.854241 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:12.854241 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:12.854241 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:12.854241 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 16:00:12.854241 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 16:00:12.854241 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 16:00:12.855169 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 16:00:12.855169 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:12.855169 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:12.855169 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:12.855169 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:12.855169 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:12.855169 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:12.855169 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:12.855169 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:12.855169 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:12.855169 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:12.855169 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:12.855169 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:12.855169 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:12.855169 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:12.855169 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:12.855169 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:12.855980 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 16:00:12.855980 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 16:00:12.855980 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 16:00:12.855980 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 16:00:12.855980 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 16:00:12.855980 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 16:00:12.855980 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:00:12.855980 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:12.860417 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 263/538] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 19 16:00:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:12.866414 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa7176d-0ca8-4b5c-b936-094591446494 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:12.935148 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:12.940799 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:12.944509 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b96f36-db4a-4fd2-b99f-dd2c7f7305d0 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:13.006631 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948050ea-4020-4ebc-8059-c4442db01865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:13.007242 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948050ea-4020-4ebc-8059-c4442db01865 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:13.007520 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948050ea-4020-4ebc-8059-c4442db01865 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:13.007794 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948050ea-4020-4ebc-8059-c4442db01865 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:13.008526 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948050ea-4020-4ebc-8059-c4442db01865 None admin] RBAC: Adding query filter params (role.id=1e475e6a170a4dbabc4ce8348eeb30b9, scope.project.id=be03e5300b3343169f35c643c93d1e06, user.id=7d596e43cc7446d29e2d62d9052d6091) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:13.008835 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948050ea-4020-4ebc-8059-c4442db01865 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:13.010277 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948050ea-4020-4ebc-8059-c4442db01865 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:13.018883 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 276/539] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 19 16:00:12 2022] GET /identity/v3/role_assignments?user.id=7d596e43cc7446d29e2d62d9052d6091&scope.project.id=be03e5300b3343169f35c643c93d1e06&role.id=1e475e6a170a4dbabc4ce8348eeb30b9 => generated 259 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:00:13.057636 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa7176d-0ca8-4b5c-b936-094591446494 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:13.125303 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9d8809-bc74-4294-98a0-713dcc058494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:13.125931 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9d8809-bc74-4294-98a0-713dcc058494 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:13.126207 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9d8809-bc74-4294-98a0-713dcc058494 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:13.126500 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9d8809-bc74-4294-98a0-713dcc058494 None admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:13.127213 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9d8809-bc74-4294-98a0-713dcc058494 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:13.127515 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9d8809-bc74-4294-98a0-713dcc058494 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:13.128968 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9d8809-bc74-4294-98a0-713dcc058494 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:13.152163 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 264/540] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 19 16:00:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:13.158664 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948050ea-4020-4ebc-8059-c4442db01865 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:13.306394 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948050ea-4020-4ebc-8059-c4442db01865 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:13.353283 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b996ba0-3c96-40e8-b96e-92cb4c021d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:13.353946 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b996ba0-3c96-40e8-b96e-92cb4c021d3d None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:13.354539 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b996ba0-3c96-40e8-b96e-92cb4c021d3d None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:13.354817 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b996ba0-3c96-40e8-b96e-92cb4c021d3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:13.355622 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b996ba0-3c96-40e8-b96e-92cb4c021d3d None admin] RBAC: Adding query filter params (role.id=1e475e6a170a4dbabc4ce8348eeb30b9, scope.project.id=be03e5300b3343169f35c643c93d1e06, user.id=50355d7b1c6c4f0bbbbc7fe56efb9039) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:13.355935 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b996ba0-3c96-40e8-b96e-92cb4c021d3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:13.357982 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b996ba0-3c96-40e8-b96e-92cb4c021d3d None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:13.366818 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 277/541] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 19 16:00:13 2022] GET /identity/v3/role_assignments?user.id=50355d7b1c6c4f0bbbbc7fe56efb9039&scope.project.id=be03e5300b3343169f35c643c93d1e06&role.id=1e475e6a170a4dbabc4ce8348eeb30b9 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:13.424106 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e59cc43-c58f-4b00-908e-178af6a9bd73 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:13.424423 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e59cc43-c58f-4b00-908e-178af6a9bd73 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:13.424652 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e59cc43-c58f-4b00-908e-178af6a9bd73 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:13.426135 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 265/542] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:00:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:00:13.432154 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0b93e4-80f8-41ea-aec1-e0815dd56a9f None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:13.432521 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0b93e4-80f8-41ea-aec1-e0815dd56a9f None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:13.432712 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0b93e4-80f8-41ea-aec1-e0815dd56a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:13.462812 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-1f0b93e4-80f8-41ea-aec1-e0815dd56a9f None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:00:13.464046 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-1f0b93e4-80f8-41ea-aec1-e0815dd56a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:00:13.495986 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:13.563729 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 278/543] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:00:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 16:00:13.818119 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba27599-9c14-4949-b92a-ae517d042987 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:13.818299 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba27599-9c14-4949-b92a-ae517d042987 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:13.818614 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba27599-9c14-4949-b92a-ae517d042987 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:13.820238 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 266/544] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:00:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:00:13.830929 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c32e054-fa34-47ef-9369-d8007fc3ad23 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:13.830929 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c32e054-fa34-47ef-9369-d8007fc3ad23 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:13.830929 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c32e054-fa34-47ef-9369-d8007fc3ad23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:13.855932 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-6c32e054-fa34-47ef-9369-d8007fc3ad23 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:00:13.857535 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-6c32e054-fa34-47ef-9369-d8007fc3ad23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:00:13.950108 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 279/545] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:00:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 16:00:13.987262 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba27599-9c14-4949-b92a-ae517d042987 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:14.116072 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba27599-9c14-4949-b92a-ae517d042987 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:14.117581 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:14.165748 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7bd267-4188-4efe-862e-49459eaaa8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:14.166659 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7bd267-4188-4efe-862e-49459eaaa8bd None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:14.166935 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7bd267-4188-4efe-862e-49459eaaa8bd None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:14.167204 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7bd267-4188-4efe-862e-49459eaaa8bd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:14.182448 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7bd267-4188-4efe-862e-49459eaaa8bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 16:00:14.188148 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-8f7bd267-4188-4efe-862e-49459eaaa8bd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 16:00:14.188148 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:14.188148 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:14.188148 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:14.188148 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:14.188148 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:14.188148 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:14.188148 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:14.188148 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:14.188148 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:14.188148 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:14.188148 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:14.188148 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 16:00:14.188148 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 16:00:14.188148 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 16:00:14.188148 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 16:00:14.189017 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 16:00:14.189017 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 16:00:14.189017 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 16:00:14.189017 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 16:00:14.189017 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:14.189017 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:14.189017 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:14.189017 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:14.189017 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:14.189017 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:14.189017 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:14.189017 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:14.189017 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:14.189017 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:14.189017 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:14.189017 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:14.189775 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:14.189775 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:14.189775 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:14.189775 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:14.189775 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 16:00:14.189775 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 16:00:14.189775 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 16:00:14.189775 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 16:00:14.189775 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 16:00:14.189775 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 16:00:14.189775 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 16:00:14.189775 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:14.192335 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 267/546] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Sep 19 16:00:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:14.198486 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c32e054-fa34-47ef-9369-d8007fc3ad23 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:14.315603 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7bd267-4188-4efe-862e-49459eaaa8bd None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:14.335505 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:14.357096 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c32e054-fa34-47ef-9369-d8007fc3ad23 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:14.422918 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7bd267-4188-4efe-862e-49459eaaa8bd None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:14.447649 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbabb861-75eb-4f2b-8b2c-ffc8fb81fb60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:14.448618 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbabb861-75eb-4f2b-8b2c-ffc8fb81fb60 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:14.449098 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbabb861-75eb-4f2b-8b2c-ffc8fb81fb60 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:14.449584 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbabb861-75eb-4f2b-8b2c-ffc8fb81fb60 None admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:14.451027 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbabb861-75eb-4f2b-8b2c-ffc8fb81fb60 None admin] RBAC: Adding query filter params (name=service) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:14.451485 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbabb861-75eb-4f2b-8b2c-ffc8fb81fb60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:14.453039 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbabb861-75eb-4f2b-8b2c-ffc8fb81fb60 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:14.460428 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f7722e-9941-405c-b662-d3fc0666106c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:14.462508 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7722e-9941-405c-b662-d3fc0666106c None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:14.462508 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7722e-9941-405c-b662-d3fc0666106c None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:14.462508 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7722e-9941-405c-b662-d3fc0666106c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:14.474332 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f7722e-9941-405c-b662-d3fc0666106c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 16:00:14.479393 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-12f7722e-9941-405c-b662-d3fc0666106c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 16:00:14.479393 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:14.479393 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:14.479393 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:14.479393 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:14.479393 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:14.479393 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:14.479393 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:14.479393 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:14.479393 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:14.479393 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:14.479393 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:14.479393 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 16:00:14.479393 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 16:00:14.479393 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 16:00:14.479393 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 16:00:14.481241 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 16:00:14.481241 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 16:00:14.481241 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 16:00:14.481241 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 16:00:14.481241 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:14.481241 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:14.481241 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:14.481241 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:14.481241 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:14.481241 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:14.481241 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:14.481241 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:14.481241 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:14.481241 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:14.481241 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:14.481241 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:14.482237 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:14.482237 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:14.482237 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:14.482237 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:14.482237 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 16:00:14.482237 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 16:00:14.482237 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 16:00:14.482237 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 16:00:14.482237 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 16:00:14.482237 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 16:00:14.482237 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 16:00:14.482237 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:14.483698 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 268/547] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Sep 19 16:00:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:14.487252 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 280/548] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Sep 19 16:00:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:14.500294 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f7722e-9941-405c-b662-d3fc0666106c None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:14.503130 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbabb861-75eb-4f2b-8b2c-ffc8fb81fb60 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:14.566400 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:14.708455 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbabb861-75eb-4f2b-8b2c-ffc8fb81fb60 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:14.724665 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f7722e-9941-405c-b662-d3fc0666106c None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:14.801590 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3301631e-6a91-4c38-86ad-a1277b7d72c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:14.802201 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301631e-6a91-4c38-86ad-a1277b7d72c6 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:14.802506 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301631e-6a91-4c38-86ad-a1277b7d72c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:14.802773 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301631e-6a91-4c38-86ad-a1277b7d72c6 None admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:14.804853 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3301631e-6a91-4c38-86ad-a1277b7d72c6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:14.805164 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3301631e-6a91-4c38-86ad-a1277b7d72c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:14.806566 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3301631e-6a91-4c38-86ad-a1277b7d72c6 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:14.809550 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bdd3d4-d13c-4701-b6b3-c99781f8f7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:14.810804 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bdd3d4-d13c-4701-b6b3-c99781f8f7c3 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:14.811086 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bdd3d4-d13c-4701-b6b3-c99781f8f7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:14.811346 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bdd3d4-d13c-4701-b6b3-c99781f8f7c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:14.827537 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bdd3d4-d13c-4701-b6b3-c99781f8f7c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:14.828998 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bdd3d4-d13c-4701-b6b3-c99781f8f7c3 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:14.831421 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 269/549] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Sep 19 16:00:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:14.836584 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3301631e-6a91-4c38-86ad-a1277b7d72c6 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:14.838891 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:14.847362 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-d1bdd3d4-d13c-4701-b6b3-c99781f8f7c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:14.847362 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:14.847362 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:14.847362 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:14.847362 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:14.847362 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:14.847362 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:14.847362 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:14.847362 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:14.847362 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:14.847362 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:14.847362 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:14.847362 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:00:14.847362 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:00:14.847362 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:00:14.847362 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:00:14.848474 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:14.848474 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:14.848474 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:14.848474 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:14.848474 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:14.848474 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:14.848474 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:14.848474 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:14.848474 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:14.848474 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:14.848474 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:14.848474 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:14.848474 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:14.848474 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:14.848474 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:14.848474 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:14.848474 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:00:14.849310 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:00:14.849310 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:14.849310 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:14.851355 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 281/550] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:00:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:14.864541 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bdd3d4-d13c-4701-b6b3-c99781f8f7c3 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:14.971291 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:14.991630 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bdd3d4-d13c-4701-b6b3-c99781f8f7c3 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:14.993053 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3301631e-6a91-4c38-86ad-a1277b7d72c6 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:15.036818 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a09a770-8201-40f8-9e0d-40f815c83eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:15.037823 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a09a770-8201-40f8-9e0d-40f815c83eec None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:15.038350 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a09a770-8201-40f8-9e0d-40f815c83eec None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:15.038851 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a09a770-8201-40f8-9e0d-40f815c83eec None admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:15.039319 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ebe674-385b-4c2b-a107-7c4ce8bad66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:15.039869 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a09a770-8201-40f8-9e0d-40f815c83eec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:15.040084 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ebe674-385b-4c2b-a107-7c4ce8bad66c None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:15.040436 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ebe674-385b-4c2b-a107-7c4ce8bad66c None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:15.040712 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a09a770-8201-40f8-9e0d-40f815c83eec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:15.040809 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ebe674-385b-4c2b-a107-7c4ce8bad66c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:15.042596 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a09a770-8201-40f8-9e0d-40f815c83eec None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:15.054848 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ebe674-385b-4c2b-a107-7c4ce8bad66c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:15.056752 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ebe674-385b-4c2b-a107-7c4ce8bad66c None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:15.058334 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 282/551] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:00:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:00:15.060508 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db73079f-82e2-4017-80c3-2efcea8b2b50 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:15.060971 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db73079f-82e2-4017-80c3-2efcea8b2b50 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:15.061414 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db73079f-82e2-4017-80c3-2efcea8b2b50 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:15.063369 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 283/552] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:00:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:00:15.064347 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db73079f-82e2-4017-80c3-2efcea8b2b50 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:15.081108 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-d6ebe674-385b-4c2b-a107-7c4ce8bad66c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:15.081108 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:15.081108 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:15.081108 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:15.081108 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:15.081108 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:15.081108 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:15.081108 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:15.081108 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:15.081108 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:15.081108 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:15.081108 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:15.081108 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:00:15.081108 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:00:15.081108 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:00:15.081108 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:00:15.082313 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:15.082313 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:15.082313 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:15.082313 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:15.082313 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:15.082313 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:15.082313 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:15.082313 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:15.082313 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:15.082313 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:15.082313 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:15.082313 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:15.082313 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:15.082313 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:15.082313 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:15.082313 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:15.082313 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:00:15.083237 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:00:15.083237 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:15.083237 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:15.085141 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 270/553] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:00:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 16:00:15.086669 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf2578a-8681-49bc-b6a9-6497c2a33701 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:15.087200 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf2578a-8681-49bc-b6a9-6497c2a33701 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:15.087675 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf2578a-8681-49bc-b6a9-6497c2a33701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:15.108380 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-caf2578a-8681-49bc-b6a9-6497c2a33701 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:00:15.109937 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-caf2578a-8681-49bc-b6a9-6497c2a33701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:00:15.135688 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:15.181468 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db73079f-82e2-4017-80c3-2efcea8b2b50 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:15.187006 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 271/554] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:00:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 16:00:15.188496 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf2578a-8681-49bc-b6a9-6497c2a33701 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:15.245146 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da1d522-b3d4-47ba-9bf9-779ec9ff790b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:15.245878 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1d522-b3d4-47ba-9bf9-779ec9ff790b None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:15.246252 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1d522-b3d4-47ba-9bf9-779ec9ff790b None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:15.246656 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1d522-b3d4-47ba-9bf9-779ec9ff790b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:15.257954 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da1d522-b3d4-47ba-9bf9-779ec9ff790b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:15.259578 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da1d522-b3d4-47ba-9bf9-779ec9ff790b None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:15.262108 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:15.286351 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-9da1d522-b3d4-47ba-9bf9-779ec9ff790b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 19 16:00:15.286351 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:15.286351 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:15.286351 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:15.286351 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:15.286351 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:15.286351 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:15.286351 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:15.286351 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:15.286351 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:15.286351 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:15.286351 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:15.286351 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 16:00:15.286351 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 16:00:15.286351 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 16:00:15.286351 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 16:00:15.287462 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:15.287462 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:15.287462 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 16:00:15.287462 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:00:15.287462 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 16:00:15.287462 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:00:15.287462 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:15.287462 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:15.287462 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:15.287462 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:15.287462 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:15.287462 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:15.287462 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:15.287462 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:15.287462 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:15.287462 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:15.287462 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:15.287462 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:15.288951 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:15.288951 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:15.288951 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 16:00:15.288951 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 16:00:15.288951 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 16:00:15.288951 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 16:00:15.288951 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 16:00:15.288951 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 16:00:15.288951 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 19 16:00:15.288951 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:15.288951 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 284/555] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 19 16:00:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:15.293415 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da1d522-b3d4-47ba-9bf9-779ec9ff790b None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:15.306797 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf2578a-8681-49bc-b6a9-6497c2a33701 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:15.345305 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a53f23-a5fb-4a12-aa5e-aa52a50c1035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:15.346479 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a53f23-a5fb-4a12-aa5e-aa52a50c1035 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:15.347080 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a53f23-a5fb-4a12-aa5e-aa52a50c1035 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:15.347563 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a53f23-a5fb-4a12-aa5e-aa52a50c1035 None admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:15.348395 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a53f23-a5fb-4a12-aa5e-aa52a50c1035 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:15.348909 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a53f23-a5fb-4a12-aa5e-aa52a50c1035 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:15.350629 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a53f23-a5fb-4a12-aa5e-aa52a50c1035 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:15.363494 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 272/556] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:00:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:15.368916 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a53f23-a5fb-4a12-aa5e-aa52a50c1035 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:15.489580 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da1d522-b3d4-47ba-9bf9-779ec9ff790b None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:15.525792 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a53f23-a5fb-4a12-aa5e-aa52a50c1035 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:15.533809 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:15.558196 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:15.570806 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6857c8-9961-454e-a6e4-b05567e82f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:15.572188 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6857c8-9961-454e-a6e4-b05567e82f90 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:15.572954 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde925f5-4763-4aed-b876-288184db9cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:15.573738 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde925f5-4763-4aed-b876-288184db9cc4 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:15.574177 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6857c8-9961-454e-a6e4-b05567e82f90 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:15.574663 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6857c8-9961-454e-a6e4-b05567e82f90 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:15.575092 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde925f5-4763-4aed-b876-288184db9cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:15.576595 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde925f5-4763-4aed-b876-288184db9cc4 None admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:15.577571 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde925f5-4763-4aed-b876-288184db9cc4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:15.578186 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde925f5-4763-4aed-b876-288184db9cc4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:15.581071 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde925f5-4763-4aed-b876-288184db9cc4 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:15.589897 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6857c8-9961-454e-a6e4-b05567e82f90 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:15.591629 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6857c8-9961-454e-a6e4-b05567e82f90 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:15.604785 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-3d6857c8-9961-454e-a6e4-b05567e82f90 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 19 16:00:15.604785 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:15.604785 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:15.604785 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:15.604785 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:15.604785 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:15.604785 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:15.604785 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:15.604785 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:15.604785 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:15.604785 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:15.604785 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:15.604785 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 16:00:15.604785 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 16:00:15.604785 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 16:00:15.604785 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 16:00:15.606478 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:15.606478 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:15.606478 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 16:00:15.606478 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:00:15.606478 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 16:00:15.606478 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:00:15.606478 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:15.606478 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:15.606478 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:15.606478 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:15.606478 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:15.606478 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:15.606478 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:15.606478 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:15.606478 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:15.606478 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:15.606478 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:15.606478 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:15.607670 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:15.607670 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:15.607670 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 16:00:15.607670 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 16:00:15.607670 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 16:00:15.607670 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 16:00:15.607670 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 16:00:15.607670 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 16:00:15.607670 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 19 16:00:15.607670 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:15.609811 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 273/557] 10.222.0.57 () {56 vars in 1093 bytes} [Mon Sep 19 16:00:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:15.620489 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6857c8-9961-454e-a6e4-b05567e82f90 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:15.652277 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 285/558] 10.222.0.57 () {56 vars in 1135 bytes} [Mon Sep 19 16:00:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:15.653965 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fd2ef2-9872-48ef-880c-827cea233c87 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:15.654350 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fd2ef2-9872-48ef-880c-827cea233c87 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:15.654648 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fd2ef2-9872-48ef-880c-827cea233c87 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:15.658663 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 286/559] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:00:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:00:15.659497 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fd2ef2-9872-48ef-880c-827cea233c87 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:15.731834 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6857c8-9961-454e-a6e4-b05567e82f90 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:15.776797 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36681e8e-9c40-4d43-a037-fa7174751b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:15.777517 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36681e8e-9c40-4d43-a037-fa7174751b9a None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:15.777783 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36681e8e-9c40-4d43-a037-fa7174751b9a None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:15.778052 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36681e8e-9c40-4d43-a037-fa7174751b9a None admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:15.778845 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36681e8e-9c40-4d43-a037-fa7174751b9a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:15.779153 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36681e8e-9c40-4d43-a037-fa7174751b9a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:15.780573 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36681e8e-9c40-4d43-a037-fa7174751b9a None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:15.781993 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fd2ef2-9872-48ef-880c-827cea233c87 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:15.800863 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 274/560] 10.222.0.57 () {56 vars in 1133 bytes} [Mon Sep 19 16:00:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:15.801527 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36681e8e-9c40-4d43-a037-fa7174751b9a None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:15.819587 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d508bd-3530-4f59-86c5-f52fba44d87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:15.820302 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d508bd-3530-4f59-86c5-f52fba44d87a None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:15.820577 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d508bd-3530-4f59-86c5-f52fba44d87a None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:15.820844 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d508bd-3530-4f59-86c5-f52fba44d87a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:15.832424 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d508bd-3530-4f59-86c5-f52fba44d87a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 16:00:15.837320 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-32d508bd-3530-4f59-86c5-f52fba44d87a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 16:00:15.837320 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:15.837320 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:15.837320 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:15.837320 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:15.837320 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:15.837320 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:15.837320 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:15.837320 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:15.837320 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:15.837320 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:15.837320 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:15.837320 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 16:00:15.837320 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 16:00:15.837320 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 16:00:15.837320 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 16:00:15.838281 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 16:00:15.838281 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 16:00:15.838281 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 16:00:15.838281 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 16:00:15.838281 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:15.838281 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:15.838281 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:15.838281 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:15.838281 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:15.838281 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:15.838281 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:15.838281 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:15.838281 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:15.838281 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:15.838281 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:15.838281 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:15.839436 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:15.839436 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:15.839436 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:15.839436 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:15.839436 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 16:00:15.839436 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 16:00:15.839436 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 16:00:15.839436 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 16:00:15.839436 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 16:00:15.839436 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 16:00:15.839436 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 16:00:15.839436 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:15.839436 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 287/561] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Sep 19 16:00:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:15.840149 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a02b886-36fb-4b56-b49b-63d287f2b583 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:15.840149 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a02b886-36fb-4b56-b49b-63d287f2b583 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:15.840149 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a02b886-36fb-4b56-b49b-63d287f2b583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:15.855052 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:15.860765 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-2a02b886-36fb-4b56-b49b-63d287f2b583 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:00:15.861974 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-2a02b886-36fb-4b56-b49b-63d287f2b583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:00:15.894806 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:15.919009 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36681e8e-9c40-4d43-a037-fa7174751b9a None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:15.937739 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 288/562] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:00:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 16:00:15.943288 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a02b886-36fb-4b56-b49b-63d287f2b583 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:15.961355 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be81b98-5b11-4673-b030-8994e36cfaf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:15.962142 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be81b98-5b11-4673-b030-8994e36cfaf2 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:15.962571 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be81b98-5b11-4673-b030-8994e36cfaf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:15.962985 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be81b98-5b11-4673-b030-8994e36cfaf2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:16.073999 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be81b98-5b11-4673-b030-8994e36cfaf2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:16.076614 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be81b98-5b11-4673-b030-8994e36cfaf2 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:16.080227 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a02b886-36fb-4b56-b49b-63d287f2b583 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:16.128394 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:16.134700 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199dadc3-b589-4ea0-9faa-f5781d958814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:16.136049 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199dadc3-b589-4ea0-9faa-f5781d958814 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:16.136519 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199dadc3-b589-4ea0-9faa-f5781d958814 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:16.136954 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199dadc3-b589-4ea0-9faa-f5781d958814 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:16.151779 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199dadc3-b589-4ea0-9faa-f5781d958814 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:16.153897 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199dadc3-b589-4ea0-9faa-f5781d958814 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:16.173011 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-199dadc3-b589-4ea0-9faa-f5781d958814 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:16.173011 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:16.173011 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:16.173011 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:16.173011 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:16.173011 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:16.173011 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:16.173011 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:16.173011 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:16.173011 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:16.173011 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:16.173011 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:16.173011 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:00:16.173011 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:00:16.173011 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:00:16.173011 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:00:16.174042 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:16.174042 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:16.174042 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:16.174042 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:16.174042 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:16.174042 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:16.174042 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:16.174042 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:16.174042 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:16.174042 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:16.174042 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:16.174042 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:16.174042 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:16.174042 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:16.174042 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:16.174042 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:16.174042 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:00:16.174965 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:00:16.174965 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:16.174965 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:16.174965 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 289/563] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:00:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:16.175202 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199dadc3-b589-4ea0-9faa-f5781d958814 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:16.204373 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 275/564] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 19 16:00:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 16:00:16.207595 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:16.211315 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0701f28-40e3-4559-b80b-db6ca6dc1552 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:16.211707 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0701f28-40e3-4559-b80b-db6ca6dc1552 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:16.211993 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0701f28-40e3-4559-b80b-db6ca6dc1552 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:16.213349 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 276/565] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:00:16 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:00:16.214273 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0701f28-40e3-4559-b80b-db6ca6dc1552 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:16.293780 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199dadc3-b589-4ea0-9faa-f5781d958814 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:16.328365 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1932091b-95bc-41f6-a5f4-0474e80d0e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:16.328828 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0701f28-40e3-4559-b80b-db6ca6dc1552 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:16.328953 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1932091b-95bc-41f6-a5f4-0474e80d0e26 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:16.329142 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1932091b-95bc-41f6-a5f4-0474e80d0e26 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:16.329412 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1932091b-95bc-41f6-a5f4-0474e80d0e26 None admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:16.330067 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1932091b-95bc-41f6-a5f4-0474e80d0e26 None admin] RBAC: Adding query filter params (name=service) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:16.330395 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1932091b-95bc-41f6-a5f4-0474e80d0e26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:16.331779 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1932091b-95bc-41f6-a5f4-0474e80d0e26 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:16.340090 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 290/566] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Sep 19 16:00:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:16.340898 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1932091b-95bc-41f6-a5f4-0474e80d0e26 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:16.366691 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f062a30-bf39-45ad-bb5c-7488238d5d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:16.367291 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f062a30-bf39-45ad-bb5c-7488238d5d56 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:16.367566 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f062a30-bf39-45ad-bb5c-7488238d5d56 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:16.367837 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f062a30-bf39-45ad-bb5c-7488238d5d56 None admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:16.368481 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f062a30-bf39-45ad-bb5c-7488238d5d56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:16.368783 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f062a30-bf39-45ad-bb5c-7488238d5d56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:16.370138 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f062a30-bf39-45ad-bb5c-7488238d5d56 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:16.382132 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 277/567] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:00:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:16.383006 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f062a30-bf39-45ad-bb5c-7488238d5d56 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:16.386514 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:16.433647 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:16.452196 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1932091b-95bc-41f6-a5f4-0474e80d0e26 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:16.487185 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f062a30-bf39-45ad-bb5c-7488238d5d56 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:16.488105 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ca8d6-a6e4-4304-b9e3-0249944f88b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:16.488742 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ca8d6-a6e4-4304-b9e3-0249944f88b2 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:16.489017 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ca8d6-a6e4-4304-b9e3-0249944f88b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:16.489287 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ca8d6-a6e4-4304-b9e3-0249944f88b2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:16.499400 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0ca8d6-a6e4-4304-b9e3-0249944f88b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 16:00:16.504573 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-7f0ca8d6-a6e4-4304-b9e3-0249944f88b2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 16:00:16.504573 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:16.504573 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:16.504573 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:16.504573 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:16.504573 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:16.504573 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:16.504573 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:16.504573 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:16.504573 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:16.504573 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:16.504573 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:16.504573 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 16:00:16.504573 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 16:00:16.504573 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 16:00:16.504573 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 16:00:16.505669 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 16:00:16.505669 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 16:00:16.505669 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 16:00:16.505669 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 16:00:16.505669 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:16.505669 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:16.505669 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:16.505669 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:16.505669 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:16.505669 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:16.505669 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:16.505669 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:16.505669 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:16.505669 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:16.505669 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:16.505669 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:16.506733 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:16.506733 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:16.506733 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:16.506733 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:16.506733 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 16:00:16.506733 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 16:00:16.506733 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 16:00:16.506733 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 16:00:16.506733 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 16:00:16.506733 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 16:00:16.506733 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 16:00:16.506733 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:16.510620 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 291/568] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Sep 19 16:00:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:16.511883 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb8964c-ba03-4c1e-b196-cf50547c1520 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:16.512194 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb8964c-ba03-4c1e-b196-cf50547c1520 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:16.512470 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb8964c-ba03-4c1e-b196-cf50547c1520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:16.520670 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d1b41-870a-4391-bf7a-9e6f70e2fa1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:16.521300 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d1b41-870a-4391-bf7a-9e6f70e2fa1e None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:16.521585 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d1b41-870a-4391-bf7a-9e6f70e2fa1e None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:16.521862 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d1b41-870a-4391-bf7a-9e6f70e2fa1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:16.530392 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-0bb8964c-ba03-4c1e-b196-cf50547c1520 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:00:16.531596 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-0bb8964c-ba03-4c1e-b196-cf50547c1520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:00:16.532960 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622d1b41-870a-4391-bf7a-9e6f70e2fa1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:16.534677 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622d1b41-870a-4391-bf7a-9e6f70e2fa1e None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:16.549625 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-622d1b41-870a-4391-bf7a-9e6f70e2fa1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:16.549625 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:16.549625 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:16.549625 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:16.549625 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:16.549625 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:16.549625 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:16.549625 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:16.549625 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:16.549625 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:16.549625 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:16.549625 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:16.549625 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:00:16.549625 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:00:16.549625 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:00:16.549625 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:00:16.550684 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:16.550684 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:16.550684 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:16.550684 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:16.550684 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:16.550684 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:16.550684 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:16.550684 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:16.550684 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:16.550684 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:16.550684 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:16.550684 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:16.550684 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:16.550684 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:16.550684 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:16.550684 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:16.550684 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:00:16.551552 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:00:16.551552 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:16.551552 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:16.551552 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 278/569] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:00:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:16.551552 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d1b41-870a-4391-bf7a-9e6f70e2fa1e None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:16.602552 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 292/570] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:00:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 16:00:16.603329 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb8964c-ba03-4c1e-b196-cf50547c1520 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:16.613102 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:16.660281 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d1b41-870a-4391-bf7a-9e6f70e2fa1e None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:16.677866 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:16.695597 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cf4b8d-1999-4757-9fe0-7dfa8fb2d4f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:16.696227 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf4b8d-1999-4757-9fe0-7dfa8fb2d4f3 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:16.696503 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf4b8d-1999-4757-9fe0-7dfa8fb2d4f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:16.696772 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf4b8d-1999-4757-9fe0-7dfa8fb2d4f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:16.707639 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cf4b8d-1999-4757-9fe0-7dfa8fb2d4f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:16.709055 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cf4b8d-1999-4757-9fe0-7dfa8fb2d4f3 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:16.710110 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb8964c-ba03-4c1e-b196-cf50547c1520 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:16.723563 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-36cf4b8d-1999-4757-9fe0-7dfa8fb2d4f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:16.723563 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:16.723563 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:16.723563 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:16.723563 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:16.723563 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:16.723563 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:16.723563 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:16.723563 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:16.723563 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:16.723563 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:16.723563 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:16.723563 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:00:16.723563 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:00:16.723563 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:00:16.723563 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:00:16.724525 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:16.724525 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:16.724525 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:16.724525 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:16.724525 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:16.724525 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:16.724525 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:16.724525 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:16.724525 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:16.724525 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:16.724525 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:16.724525 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:16.724525 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:16.724525 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:16.724525 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:16.724525 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:16.724525 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:00:16.725399 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:00:16.725399 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:16.725399 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:16.725399 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 279/571] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:00:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:16.725399 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cf4b8d-1999-4757-9fe0-7dfa8fb2d4f3 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:16.751129 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9346a12-9eed-4795-9bfa-f9bd4abd5aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:16.751782 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9346a12-9eed-4795-9bfa-f9bd4abd5aa2 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:16.752034 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9346a12-9eed-4795-9bfa-f9bd4abd5aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:16.752304 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9346a12-9eed-4795-9bfa-f9bd4abd5aa2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:16.762348 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9346a12-9eed-4795-9bfa-f9bd4abd5aa2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:16.762348 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9346a12-9eed-4795-9bfa-f9bd4abd5aa2 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:16.775776 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-b9346a12-9eed-4795-9bfa-f9bd4abd5aa2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:00:16.775776 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:16.775776 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:16.775776 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:16.775776 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:16.775776 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:16.775776 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:16.775776 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:16.775776 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:16.775776 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:16.775776 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:16.775776 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:16.775776 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 16:00:16.775776 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 16:00:16.775776 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 16:00:16.776902 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 16:00:16.776902 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:16.776902 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:16.776902 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:16.776902 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:16.776902 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:16.776902 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:16.776902 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:16.776902 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:16.776902 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:16.776902 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:16.776902 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:16.776902 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:16.776902 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:16.776902 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:16.776902 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:16.776902 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:16.777999 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 16:00:16.777999 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 16:00:16.777999 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 16:00:16.777999 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 16:00:16.777999 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 16:00:16.777999 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 16:00:16.777999 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:00:16.777999 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:16.777999 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 293/572] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 19 16:00:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:16.777999 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9346a12-9eed-4795-9bfa-f9bd4abd5aa2 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:16.850302 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cf4b8d-1999-4757-9fe0-7dfa8fb2d4f3 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:16.861297 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:16.890074 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9346a12-9eed-4795-9bfa-f9bd4abd5aa2 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:16.891804 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b18831-4583-4ca6-aecc-3fa44420b1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:16.892476 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b18831-4583-4ca6-aecc-3fa44420b1f9 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:16.892762 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b18831-4583-4ca6-aecc-3fa44420b1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:16.893035 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b18831-4583-4ca6-aecc-3fa44420b1f9 None admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:16.893699 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b18831-4583-4ca6-aecc-3fa44420b1f9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:16.894014 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b18831-4583-4ca6-aecc-3fa44420b1f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:16.895404 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b18831-4583-4ca6-aecc-3fa44420b1f9 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:16.903725 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 280/573] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Sep 19 16:00:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:16.904538 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b18831-4583-4ca6-aecc-3fa44420b1f9 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:16.927270 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:16.932658 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470bd6b3-d093-4400-80b1-c025b1b02063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:16.933266 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470bd6b3-d093-4400-80b1-c025b1b02063 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:16.933545 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470bd6b3-d093-4400-80b1-c025b1b02063 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:16.933819 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470bd6b3-d093-4400-80b1-c025b1b02063 None admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:16.934501 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470bd6b3-d093-4400-80b1-c025b1b02063 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:16.936414 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470bd6b3-d093-4400-80b1-c025b1b02063 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:16.937828 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470bd6b3-d093-4400-80b1-c025b1b02063 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:16.949805 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 294/574] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:00:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:16.950677 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470bd6b3-d093-4400-80b1-c025b1b02063 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:17.008023 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b18831-4583-4ca6-aecc-3fa44420b1f9 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:17.044840 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426affa0-e1d7-42a7-88d2-afc17fb1976a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:17.045802 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426affa0-e1d7-42a7-88d2-afc17fb1976a None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:17.046175 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426affa0-e1d7-42a7-88d2-afc17fb1976a None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:17.046666 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426affa0-e1d7-42a7-88d2-afc17fb1976a None admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:17.047761 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426affa0-e1d7-42a7-88d2-afc17fb1976a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:17.048273 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426affa0-e1d7-42a7-88d2-afc17fb1976a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:17.050594 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426affa0-e1d7-42a7-88d2-afc17fb1976a None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:17.057292 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470bd6b3-d093-4400-80b1-c025b1b02063 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:17.063869 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 281/575] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:00:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:17.064717 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426affa0-e1d7-42a7-88d2-afc17fb1976a None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:17.091913 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161518f5-c0b5-471e-834e-ade8c81f9616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:17.092524 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161518f5-c0b5-471e-834e-ade8c81f9616 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:17.092798 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161518f5-c0b5-471e-834e-ade8c81f9616 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:17.093064 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161518f5-c0b5-471e-834e-ade8c81f9616 None admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:17.093749 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161518f5-c0b5-471e-834e-ade8c81f9616 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:17.094053 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161518f5-c0b5-471e-834e-ade8c81f9616 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:17.094479 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:17.095552 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161518f5-c0b5-471e-834e-ade8c81f9616 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:17.108060 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 295/576] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 19 16:00:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:17.111355 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161518f5-c0b5-471e-834e-ade8c81f9616 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:17.161606 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:17.164691 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426affa0-e1d7-42a7-88d2-afc17fb1976a None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:17.199458 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d274328e-55f6-433d-9563-8fab8746085e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:17.200083 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d274328e-55f6-433d-9563-8fab8746085e None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:17.200410 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d274328e-55f6-433d-9563-8fab8746085e None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:17.200691 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d274328e-55f6-433d-9563-8fab8746085e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:17.209388 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161518f5-c0b5-471e-834e-ade8c81f9616 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:17.210941 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d274328e-55f6-433d-9563-8fab8746085e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 16:00:17.215725 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-d274328e-55f6-433d-9563-8fab8746085e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 16:00:17.215725 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:17.215725 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:17.215725 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:17.215725 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:17.215725 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:17.215725 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:17.215725 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:17.215725 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:17.215725 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:17.215725 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:17.215725 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:17.215725 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 16:00:17.215725 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 16:00:17.215725 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 16:00:17.215725 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 16:00:17.217260 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 16:00:17.217260 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 16:00:17.217260 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 16:00:17.217260 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 16:00:17.217260 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:17.217260 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:17.217260 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:17.217260 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:17.217260 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:17.217260 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:17.217260 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:17.217260 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:17.217260 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:17.217260 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:17.217260 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:17.217260 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:17.218064 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:17.218064 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:17.218064 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:17.218064 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:17.218064 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 16:00:17.218064 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 16:00:17.218064 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 16:00:17.218064 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 16:00:17.218064 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 16:00:17.218064 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 16:00:17.218064 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 16:00:17.218064 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:17.218064 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 282/577] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Sep 19 16:00:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:17.219473 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d274328e-55f6-433d-9563-8fab8746085e None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:17.246638 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfade0f-e1b7-4c2d-957d-a443b931976b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:17.247268 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfade0f-e1b7-4c2d-957d-a443b931976b None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:17.247541 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfade0f-e1b7-4c2d-957d-a443b931976b None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:17.247808 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfade0f-e1b7-4c2d-957d-a443b931976b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:17.259388 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfade0f-e1b7-4c2d-957d-a443b931976b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:17.260818 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfade0f-e1b7-4c2d-957d-a443b931976b None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:17.276558 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-1cfade0f-e1b7-4c2d-957d-a443b931976b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:17.276558 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:17.276558 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:17.276558 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:17.276558 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:17.276558 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:17.276558 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:17.276558 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:17.276558 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:17.276558 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:17.276558 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:17.276558 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:17.276558 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:00:17.276558 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:00:17.276558 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:00:17.276558 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:00:17.277735 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:17.277735 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:17.277735 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:17.277735 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:17.277735 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:17.277735 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:17.277735 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:17.277735 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:17.277735 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:17.277735 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:17.277735 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:17.277735 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:17.277735 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:17.277735 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:17.277735 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:17.277735 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:17.277735 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:00:17.279146 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:00:17.279146 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:17.279146 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:17.279146 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 296/578] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:00:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:17.279146 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfade0f-e1b7-4c2d-957d-a443b931976b None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:17.318283 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:17.329043 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d274328e-55f6-433d-9563-8fab8746085e None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:17.364872 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f792d035-a41e-4fde-af63-859181a2eecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:17.365425 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f792d035-a41e-4fde-af63-859181a2eecc None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:17.365703 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f792d035-a41e-4fde-af63-859181a2eecc None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:17.365974 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f792d035-a41e-4fde-af63-859181a2eecc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:17.372920 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f792d035-a41e-4fde-af63-859181a2eecc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:17.374342 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f792d035-a41e-4fde-af63-859181a2eecc None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:17.380288 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfade0f-e1b7-4c2d-957d-a443b931976b None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:17.382638 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:17.385545 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-f792d035-a41e-4fde-af63-859181a2eecc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:00:17.385545 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:17.385545 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:17.385545 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:17.385545 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:17.385545 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:17.385545 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:17.385545 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:17.385545 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:17.385545 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:17.385545 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:17.385545 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:17.385545 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 16:00:17.385545 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 16:00:17.385545 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 16:00:17.386635 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 16:00:17.386635 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:17.386635 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:17.386635 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:17.386635 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:17.386635 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:17.386635 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:17.386635 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:17.386635 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:17.386635 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:17.386635 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:17.386635 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:17.386635 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:17.386635 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:17.386635 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:17.386635 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:17.386635 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:17.387526 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 16:00:17.387526 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 16:00:17.387526 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 16:00:17.387526 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 16:00:17.387526 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 16:00:17.387526 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 16:00:17.387526 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:00:17.387526 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:17.387526 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 283/579] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 19 16:00:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:17.387526 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f792d035-a41e-4fde-af63-859181a2eecc None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:17.417909 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b7e8b9-7ec7-4a9e-93d4-406830e30891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:17.418576 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b7e8b9-7ec7-4a9e-93d4-406830e30891 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:17.418859 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b7e8b9-7ec7-4a9e-93d4-406830e30891 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:17.419757 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b7e8b9-7ec7-4a9e-93d4-406830e30891 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:17.427115 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b7e8b9-7ec7-4a9e-93d4-406830e30891 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:17.428584 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b7e8b9-7ec7-4a9e-93d4-406830e30891 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:17.440067 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-41b7e8b9-7ec7-4a9e-93d4-406830e30891 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 19 16:00:17.440067 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:17.440067 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:17.440067 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:17.440067 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:17.440067 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:17.440067 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:17.440067 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:17.440067 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:17.440067 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:17.440067 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:17.440067 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:17.440067 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 16:00:17.440067 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 16:00:17.440067 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 16:00:17.440067 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 16:00:17.441055 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:17.441055 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:17.441055 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 16:00:17.441055 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:00:17.441055 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 16:00:17.441055 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:00:17.441055 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:17.441055 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:17.441055 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:17.441055 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:17.441055 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:17.441055 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:17.441055 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:17.441055 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:17.441055 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:17.441055 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:17.441055 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:17.441055 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:17.442015 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:17.442015 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:17.442015 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 16:00:17.442015 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 16:00:17.442015 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 16:00:17.442015 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 16:00:17.442015 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 16:00:17.442015 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 16:00:17.442015 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 19 16:00:17.442015 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:17.442015 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 297/580] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 16:00:17 2022] GET /identity/v3/users/glance => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:17.442015 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b7e8b9-7ec7-4a9e-93d4-406830e30891 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:17.491715 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f792d035-a41e-4fde-af63-859181a2eecc None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:17.524710 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71e6ed4-b6fa-48a7-b5f4-86622939f5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:17.525360 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71e6ed4-b6fa-48a7-b5f4-86622939f5d4 None admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:17.525636 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71e6ed4-b6fa-48a7-b5f4-86622939f5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:17.525909 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71e6ed4-b6fa-48a7-b5f4-86622939f5d4 None admin] PATH_INFO: `/v3/projects/be03e5300b3343169f35c643c93d1e06/users/1ecd01cc72834b72b5ff2c4b47249737/roles/1e475e6a170a4dbabc4ce8348eeb30b9` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:17.539568 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b7e8b9-7ec7-4a9e-93d4-406830e30891 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:17.540485 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71e6ed4-b6fa-48a7-b5f4-86622939f5d4 None admin] RBAC: Authorizing `identity:create_grant(project_id=be03e5300b3343169f35c643c93d1e06, user_id=1ecd01cc72834b72b5ff2c4b47249737, role_id=1e475e6a170a4dbabc4ce8348eeb30b9)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:17.542159 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71e6ed4-b6fa-48a7-b5f4-86622939f5d4 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:17.552250 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:17.556959 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 284/581] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Sep 19 16:00:17 2022] PUT /identity/v3/projects/be03e5300b3343169f35c643c93d1e06/users/1ecd01cc72834b72b5ff2c4b47249737/roles/1e475e6a170a4dbabc4ce8348eeb30b9 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:00:17.558228 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71e6ed4-b6fa-48a7-b5f4-86622939f5d4 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:17.580866 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7bd32a-399b-46a0-8ad7-881f28cd1bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:17.581491 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7bd32a-399b-46a0-8ad7-881f28cd1bb8 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:17.581774 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7bd32a-399b-46a0-8ad7-881f28cd1bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:17.582050 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7bd32a-399b-46a0-8ad7-881f28cd1bb8 None admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:17.582779 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7bd32a-399b-46a0-8ad7-881f28cd1bb8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:17.583105 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7bd32a-399b-46a0-8ad7-881f28cd1bb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:17.584528 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7bd32a-399b-46a0-8ad7-881f28cd1bb8 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:17.598916 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 298/582] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Sep 19 16:00:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:17.598916 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7bd32a-399b-46a0-8ad7-881f28cd1bb8 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:17.627630 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:17.681485 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71e6ed4-b6fa-48a7-b5f4-86622939f5d4 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:17.711770 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7bd32a-399b-46a0-8ad7-881f28cd1bb8 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:17.722752 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9ab978-cbd5-409f-b681-9531a889b9ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:17.723372 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ab978-cbd5-409f-b681-9531a889b9ea None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:17.723655 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ab978-cbd5-409f-b681-9531a889b9ea None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:17.723931 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ab978-cbd5-409f-b681-9531a889b9ea None admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:17.724582 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9ab978-cbd5-409f-b681-9531a889b9ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:17.724884 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9ab978-cbd5-409f-b681-9531a889b9ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:17.726253 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9ab978-cbd5-409f-b681-9531a889b9ea None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:17.743187 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 285/583] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:00:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:17.743964 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9ab978-cbd5-409f-b681-9531a889b9ea None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:17.752567 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb5438c-59ed-464d-957c-536cb7faf23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:17.753179 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5438c-59ed-464d-957c-536cb7faf23b None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:17.753453 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5438c-59ed-464d-957c-536cb7faf23b None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:17.753729 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5438c-59ed-464d-957c-536cb7faf23b None admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:17.754456 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb5438c-59ed-464d-957c-536cb7faf23b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:17.754788 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb5438c-59ed-464d-957c-536cb7faf23b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:17.756252 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb5438c-59ed-464d-957c-536cb7faf23b None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:17.769301 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 299/584] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 19 16:00:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:17.770154 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb5438c-59ed-464d-957c-536cb7faf23b None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:17.823539 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:17.866060 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:17.876026 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9ab978-cbd5-409f-b681-9531a889b9ea None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:17.883859 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb5438c-59ed-464d-957c-536cb7faf23b None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:17.912380 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4360d332-9e47-4fdc-84d6-1fbcf1afdc52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:17.912983 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4360d332-9e47-4fdc-84d6-1fbcf1afdc52 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:17.913255 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4360d332-9e47-4fdc-84d6-1fbcf1afdc52 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:17.913768 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4360d332-9e47-4fdc-84d6-1fbcf1afdc52 None admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:17.914230 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4360d332-9e47-4fdc-84d6-1fbcf1afdc52 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:17.914564 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4360d332-9e47-4fdc-84d6-1fbcf1afdc52 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:17.915966 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4360d332-9e47-4fdc-84d6-1fbcf1afdc52 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:17.923323 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0697051f-175b-452d-a38b-70c7f275d304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:17.924031 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0697051f-175b-452d-a38b-70c7f275d304 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:17.924306 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0697051f-175b-452d-a38b-70c7f275d304 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:17.924582 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0697051f-175b-452d-a38b-70c7f275d304 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:17.935839 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0697051f-175b-452d-a38b-70c7f275d304 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:17.937067 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 286/585] 10.222.0.57 () {56 vars in 1137 bytes} [Mon Sep 19 16:00:17 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:17.937297 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0697051f-175b-452d-a38b-70c7f275d304 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:17.942308 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4360d332-9e47-4fdc-84d6-1fbcf1afdc52 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:17.954253 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-0697051f-175b-452d-a38b-70c7f275d304 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:17.954253 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:17.954253 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:17.954253 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:17.954253 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:17.954253 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:17.954253 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:17.954253 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:17.954253 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:17.954253 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:17.954253 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:17.954253 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:17.954253 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:00:17.954253 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:00:17.954253 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:00:17.954253 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:00:17.955235 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:17.955235 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:17.955235 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:17.955235 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:17.955235 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:17.955235 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:17.955235 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:17.955235 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:17.955235 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:17.955235 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:17.955235 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:17.955235 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:17.955235 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:17.955235 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:17.955235 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:17.955235 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:17.955235 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:00:17.956098 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:00:17.956098 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:17.956098 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:17.960087 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 300/586] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:00:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:17.960758 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0697051f-175b-452d-a38b-70c7f275d304 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:18.052467 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4360d332-9e47-4fdc-84d6-1fbcf1afdc52 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:18.065955 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0697051f-175b-452d-a38b-70c7f275d304 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:18.071653 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:18.091777 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb170b6-8d46-412d-abb5-6f3b8c001b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:18.092588 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb170b6-8d46-412d-abb5-6f3b8c001b54 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:18.092853 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb170b6-8d46-412d-abb5-6f3b8c001b54 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:18.093188 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb170b6-8d46-412d-abb5-6f3b8c001b54 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:18.101148 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb170b6-8d46-412d-abb5-6f3b8c001b54 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:18.102278 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78415498-adf5-462f-be38-3c080ee34f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:18.102692 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb170b6-8d46-412d-abb5-6f3b8c001b54 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:18.102913 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78415498-adf5-462f-be38-3c080ee34f0c None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:18.103220 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78415498-adf5-462f-be38-3c080ee34f0c None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:18.103464 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78415498-adf5-462f-be38-3c080ee34f0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:18.104188 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78415498-adf5-462f-be38-3c080ee34f0c None admin] RBAC: Adding query filter params (role.id=1e475e6a170a4dbabc4ce8348eeb30b9, scope.project.id=be03e5300b3343169f35c643c93d1e06, user.id=884a0524aca744b99c7496c11c3291ae) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:18.104496 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78415498-adf5-462f-be38-3c080ee34f0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:18.105908 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78415498-adf5-462f-be38-3c080ee34f0c None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:18.112149 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 301/587] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 19 16:00:17 2022] GET /identity/v3/role_assignments?user.id=884a0524aca744b99c7496c11c3291ae&scope.project.id=be03e5300b3343169f35c643c93d1e06&role.id=1e475e6a170a4dbabc4ce8348eeb30b9 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:18.112962 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78415498-adf5-462f-be38-3c080ee34f0c None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:18.116284 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-fbb170b6-8d46-412d-abb5-6f3b8c001b54 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 19 16:00:18.116284 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:18.116284 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:18.116284 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:18.116284 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:18.116284 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:18.116284 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:18.116284 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:18.116284 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:18.116284 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:18.116284 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:18.116284 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:18.116284 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 16:00:18.116284 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 16:00:18.116284 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 16:00:18.116284 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 16:00:18.117275 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:18.117275 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:18.117275 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 16:00:18.117275 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:00:18.117275 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 16:00:18.117275 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:00:18.117275 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:18.117275 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:18.117275 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:18.117275 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:18.117275 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:18.117275 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:18.117275 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:18.117275 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:18.117275 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:18.117275 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:18.117275 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:18.117275 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:18.118193 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:18.118193 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:18.118193 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 16:00:18.118193 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 16:00:18.118193 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 16:00:18.118193 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 16:00:18.118193 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 16:00:18.118193 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 16:00:18.118193 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 19 16:00:18.118193 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:18.118193 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 287/588] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 16:00:17 2022] GET /identity/v3/users/cinder => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:18.119273 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb170b6-8d46-412d-abb5-6f3b8c001b54 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:18.121684 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:18.249496 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb170b6-8d46-412d-abb5-6f3b8c001b54 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:18.251966 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78415498-adf5-462f-be38-3c080ee34f0c None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:18.295876 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726dc561-be12-4ad6-9683-7747047d26c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:18.296500 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726dc561-be12-4ad6-9683-7747047d26c5 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:18.296775 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726dc561-be12-4ad6-9683-7747047d26c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:18.297043 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726dc561-be12-4ad6-9683-7747047d26c5 None admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:18.297898 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726dc561-be12-4ad6-9683-7747047d26c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:18.298201 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726dc561-be12-4ad6-9683-7747047d26c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:18.300399 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726dc561-be12-4ad6-9683-7747047d26c5 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:18.305069 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bf8e04-c6c2-4d75-9ae7-f46546989ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:18.305991 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bf8e04-c6c2-4d75-9ae7-f46546989ddd None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:18.306468 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bf8e04-c6c2-4d75-9ae7-f46546989ddd None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:18.306895 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bf8e04-c6c2-4d75-9ae7-f46546989ddd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:18.319670 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 288/589] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:00:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:18.319670 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726dc561-be12-4ad6-9683-7747047d26c5 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:18.332396 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bf8e04-c6c2-4d75-9ae7-f46546989ddd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:18.333754 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bf8e04-c6c2-4d75-9ae7-f46546989ddd None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:18.351826 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-a2bf8e04-c6c2-4d75-9ae7-f46546989ddd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:18.351826 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:18.351826 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:18.351826 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:18.351826 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:18.351826 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:18.351826 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:18.351826 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:18.351826 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:18.351826 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:18.351826 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:18.351826 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:18.351826 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:00:18.351826 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:00:18.351826 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:00:18.351826 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:00:18.352845 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:18.352845 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:18.352845 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:18.352845 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:18.352845 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:18.352845 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:18.352845 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:18.352845 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:18.352845 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:18.352845 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:18.352845 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:18.352845 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:18.352845 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:18.352845 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:18.352845 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:18.352845 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:18.352845 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:00:18.353723 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:00:18.353723 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:18.353723 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:18.356199 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 302/590] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:00:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:18.357898 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bf8e04-c6c2-4d75-9ae7-f46546989ddd None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:18.373196 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:18.423693 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:18.441194 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726dc561-be12-4ad6-9683-7747047d26c5 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:18.478911 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bf8e04-c6c2-4d75-9ae7-f46546989ddd None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:18.485007 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0bc1e0-2bcb-4447-83e7-7bcf8d5efc7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:18.485634 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0bc1e0-2bcb-4447-83e7-7bcf8d5efc7a None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:18.485912 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0bc1e0-2bcb-4447-83e7-7bcf8d5efc7a None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:18.486181 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0bc1e0-2bcb-4447-83e7-7bcf8d5efc7a None admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:18.486950 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0bc1e0-2bcb-4447-83e7-7bcf8d5efc7a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:18.487257 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0bc1e0-2bcb-4447-83e7-7bcf8d5efc7a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:18.488690 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0bc1e0-2bcb-4447-83e7-7bcf8d5efc7a None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:18.514629 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 289/591] 10.222.0.57 () {56 vars in 1137 bytes} [Mon Sep 19 16:00:18 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:18.516271 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0bc1e0-2bcb-4447-83e7-7bcf8d5efc7a None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:18.528244 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3380714-c186-45b1-8635-835a09f26029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:18.529512 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3380714-c186-45b1-8635-835a09f26029 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:18.529864 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3380714-c186-45b1-8635-835a09f26029 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:18.530208 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3380714-c186-45b1-8635-835a09f26029 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:18.541217 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3380714-c186-45b1-8635-835a09f26029 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:18.543055 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3380714-c186-45b1-8635-835a09f26029 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:18.559600 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-a3380714-c186-45b1-8635-835a09f26029 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 19 16:00:18.559600 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:18.559600 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:18.559600 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:18.559600 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:18.559600 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:18.559600 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:18.559600 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:18.559600 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:18.559600 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:18.559600 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:18.559600 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:18.559600 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 16:00:18.559600 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 16:00:18.559600 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 16:00:18.559600 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 16:00:18.561210 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:18.561210 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:18.561210 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 16:00:18.561210 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:00:18.561210 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 16:00:18.561210 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:00:18.561210 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:18.561210 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:18.561210 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:18.561210 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:18.561210 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:18.561210 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:18.561210 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:18.561210 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:18.561210 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:18.561210 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:18.561210 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:18.561210 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:18.562106 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:18.562106 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:18.562106 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 16:00:18.562106 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 16:00:18.562106 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 16:00:18.562106 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 16:00:18.562106 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 16:00:18.562106 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 16:00:18.562106 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 19 16:00:18.562106 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:18.566844 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 303/592] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Sep 19 16:00:18 2022] GET /identity/v3/users/neutron => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:18.569053 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3380714-c186-45b1-8635-835a09f26029 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:18.641652 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:18.650836 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0bc1e0-2bcb-4447-83e7-7bcf8d5efc7a None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:18.698811 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c4ce25-c4b7-4301-a941-b566212ba21e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:18.699531 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c4ce25-c4b7-4301-a941-b566212ba21e None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:18.699879 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c4ce25-c4b7-4301-a941-b566212ba21e None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:18.700422 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c4ce25-c4b7-4301-a941-b566212ba21e None admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:18.701085 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c4ce25-c4b7-4301-a941-b566212ba21e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:18.701920 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c4ce25-c4b7-4301-a941-b566212ba21e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:18.703197 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c4ce25-c4b7-4301-a941-b566212ba21e None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:18.708932 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3380714-c186-45b1-8635-835a09f26029 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:18.718564 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 290/593] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:00:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:18.719712 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c4ce25-c4b7-4301-a941-b566212ba21e None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:18.727564 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:18.754035 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03090fa7-76a3-49a1-ba72-f49350d42a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:18.754899 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03090fa7-76a3-49a1-ba72-f49350d42a04 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:18.755309 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03090fa7-76a3-49a1-ba72-f49350d42a04 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:18.755688 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03090fa7-76a3-49a1-ba72-f49350d42a04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:18.772017 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03090fa7-76a3-49a1-ba72-f49350d42a04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:18.775839 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03090fa7-76a3-49a1-ba72-f49350d42a04 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:18.799077 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-03090fa7-76a3-49a1-ba72-f49350d42a04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:18.799077 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:18.799077 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:18.799077 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:18.799077 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:18.799077 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:18.799077 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:18.799077 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:18.799077 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:18.799077 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:18.799077 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:18.799077 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:18.799077 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:00:18.799077 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:00:18.799077 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:00:18.799077 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:00:18.800255 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:18.800255 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:18.800255 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:18.800255 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:18.800255 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:18.800255 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:18.800255 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:18.800255 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:18.800255 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:18.800255 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:18.800255 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:18.800255 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:18.800255 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:18.800255 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:18.800255 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:18.800255 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:18.800255 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:00:18.801337 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:00:18.801337 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:18.801337 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:18.803518 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 304/594] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:00:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:18.803835 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03090fa7-76a3-49a1-ba72-f49350d42a04 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:18.846524 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c4ce25-c4b7-4301-a941-b566212ba21e None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:18.885530 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac5abfa-a05c-4486-b303-365e9d9942b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:18.886207 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac5abfa-a05c-4486-b303-365e9d9942b1 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:18.887107 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac5abfa-a05c-4486-b303-365e9d9942b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:18.887443 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac5abfa-a05c-4486-b303-365e9d9942b1 None admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:18.888870 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac5abfa-a05c-4486-b303-365e9d9942b1 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:18.889240 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac5abfa-a05c-4486-b303-365e9d9942b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:18.890796 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac5abfa-a05c-4486-b303-365e9d9942b1 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:18.910228 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 291/595] 10.222.0.57 () {56 vars in 1139 bytes} [Mon Sep 19 16:00:18 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:18.911232 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac5abfa-a05c-4486-b303-365e9d9942b1 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:18.933366 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:18.956264 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03090fa7-76a3-49a1-ba72-f49350d42a04 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:19.001485 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1dec99-dab5-4462-96e5-8b6c7e0f2abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:19.002770 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1dec99-dab5-4462-96e5-8b6c7e0f2abf None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:19.003762 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1dec99-dab5-4462-96e5-8b6c7e0f2abf None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:19.004553 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1dec99-dab5-4462-96e5-8b6c7e0f2abf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:19.015209 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1dec99-dab5-4462-96e5-8b6c7e0f2abf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:19.017711 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1dec99-dab5-4462-96e5-8b6c7e0f2abf None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:19.021729 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac5abfa-a05c-4486-b303-365e9d9942b1 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:19.033948 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-fd1dec99-dab5-4462-96e5-8b6c7e0f2abf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:00:19.033948 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:19.033948 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:19.033948 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:19.033948 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:19.033948 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:19.033948 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:19.033948 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:19.033948 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:19.033948 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:19.033948 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:19.033948 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:19.033948 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 16:00:19.033948 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 16:00:19.033948 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 16:00:19.035194 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 16:00:19.035194 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:19.035194 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:19.035194 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:19.035194 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:19.035194 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:19.035194 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:19.035194 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:19.035194 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:19.035194 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:19.035194 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:19.035194 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:19.035194 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:19.035194 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:19.035194 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:19.035194 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:19.035194 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:19.036055 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 16:00:19.036055 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 16:00:19.036055 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 16:00:19.036055 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 16:00:19.036055 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 16:00:19.036055 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 16:00:19.036055 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:00:19.036055 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:19.036055 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 305/596] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 19 16:00:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:19.037072 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1dec99-dab5-4462-96e5-8b6c7e0f2abf None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:19.064025 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8e1334-8e24-4a77-8ed7-6030b96f9b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:19.064631 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8e1334-8e24-4a77-8ed7-6030b96f9b8f None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:19.064956 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8e1334-8e24-4a77-8ed7-6030b96f9b8f None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:19.065279 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8e1334-8e24-4a77-8ed7-6030b96f9b8f None admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:19.065921 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8e1334-8e24-4a77-8ed7-6030b96f9b8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:19.066229 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8e1334-8e24-4a77-8ed7-6030b96f9b8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:19.067681 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8e1334-8e24-4a77-8ed7-6030b96f9b8f None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:19.075134 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:19.080723 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 292/597] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:00:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:19.081621 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8e1334-8e24-4a77-8ed7-6030b96f9b8f None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:19.167567 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1dec99-dab5-4462-96e5-8b6c7e0f2abf None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:19.185675 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:19.202992 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553d55b9-d298-4954-bf11-c5f2121ffc1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:19.203619 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d55b9-d298-4954-bf11-c5f2121ffc1b None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:19.203894 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d55b9-d298-4954-bf11-c5f2121ffc1b None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:19.204164 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d55b9-d298-4954-bf11-c5f2121ffc1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:19.216886 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553d55b9-d298-4954-bf11-c5f2121ffc1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:19.218482 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553d55b9-d298-4954-bf11-c5f2121ffc1b None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:19.221390 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8e1334-8e24-4a77-8ed7-6030b96f9b8f None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:19.233730 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-553d55b9-d298-4954-bf11-c5f2121ffc1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:19.233730 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:19.233730 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:19.233730 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:19.233730 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:19.233730 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:19.233730 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:19.233730 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:19.233730 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:19.233730 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:19.233730 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:19.233730 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:19.233730 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:00:19.233730 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:00:19.233730 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:00:19.233730 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:00:19.234987 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:19.234987 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:19.234987 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:19.234987 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:19.234987 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:19.234987 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:19.234987 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:19.234987 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:19.234987 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:19.234987 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:19.234987 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:19.234987 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:19.234987 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:19.234987 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:19.234987 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:19.234987 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:19.234987 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:00:19.235958 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:00:19.235958 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:19.235958 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:19.235958 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 306/598] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:00:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:19.236190 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553d55b9-d298-4954-bf11-c5f2121ffc1b None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:19.268657 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4c429d-5e17-4b39-a42c-b137e4ad09a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:19.269230 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4c429d-5e17-4b39-a42c-b137e4ad09a4 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:19.269432 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4c429d-5e17-4b39-a42c-b137e4ad09a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:19.269698 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4c429d-5e17-4b39-a42c-b137e4ad09a4 None admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:19.270710 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4c429d-5e17-4b39-a42c-b137e4ad09a4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:19.270831 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4c429d-5e17-4b39-a42c-b137e4ad09a4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:19.272268 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4c429d-5e17-4b39-a42c-b137e4ad09a4 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:19.292773 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 293/599] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 19 16:00:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:19.292773 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4c429d-5e17-4b39-a42c-b137e4ad09a4 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:19.347957 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:19.371580 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553d55b9-d298-4954-bf11-c5f2121ffc1b None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:19.408034 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af999d2e-9a0b-41c9-8a34-db6901b8db10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:19.408665 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af999d2e-9a0b-41c9-8a34-db6901b8db10 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:19.408968 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af999d2e-9a0b-41c9-8a34-db6901b8db10 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:19.409621 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af999d2e-9a0b-41c9-8a34-db6901b8db10 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:19.417262 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af999d2e-9a0b-41c9-8a34-db6901b8db10 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:19.418947 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af999d2e-9a0b-41c9-8a34-db6901b8db10 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:19.424479 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4c429d-5e17-4b39-a42c-b137e4ad09a4 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:19.431114 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-af999d2e-9a0b-41c9-8a34-db6901b8db10 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:00:19.431114 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:19.431114 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:19.431114 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:19.431114 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:19.431114 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:19.431114 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:19.431114 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:19.431114 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:19.431114 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:19.431114 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:19.431114 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:19.431114 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 16:00:19.431114 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 16:00:19.431114 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 16:00:19.432336 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 16:00:19.432336 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:19.432336 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:19.432336 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:19.432336 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:19.432336 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:19.432336 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:19.432336 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:19.432336 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:19.432336 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:19.432336 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:19.432336 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:19.432336 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:19.432336 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:19.432336 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:19.432336 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:19.432336 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:19.433218 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 16:00:19.433218 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 16:00:19.433218 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 16:00:19.433218 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 16:00:19.433218 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 16:00:19.433218 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 16:00:19.433218 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:00:19.433218 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:19.433218 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 307/600] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 19 16:00:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:19.433708 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af999d2e-9a0b-41c9-8a34-db6901b8db10 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:19.467232 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2067dc5-9478-4cff-8a18-7618b5aa5de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:19.468174 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2067dc5-9478-4cff-8a18-7618b5aa5de0 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:19.468556 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2067dc5-9478-4cff-8a18-7618b5aa5de0 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:19.468922 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2067dc5-9478-4cff-8a18-7618b5aa5de0 None admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:19.470058 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2067dc5-9478-4cff-8a18-7618b5aa5de0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:19.470548 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2067dc5-9478-4cff-8a18-7618b5aa5de0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:19.472056 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2067dc5-9478-4cff-8a18-7618b5aa5de0 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:19.473857 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:19.486747 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 294/601] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:00:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:19.488116 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2067dc5-9478-4cff-8a18-7618b5aa5de0 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:19.567626 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af999d2e-9a0b-41c9-8a34-db6901b8db10 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:19.605299 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2067dc5-9478-4cff-8a18-7618b5aa5de0 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:19.607627 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d42461-0926-45f7-9ad8-e5fc0360de2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:19.609013 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d42461-0926-45f7-9ad8-e5fc0360de2a None admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:19.609013 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d42461-0926-45f7-9ad8-e5fc0360de2a None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:19.609013 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d42461-0926-45f7-9ad8-e5fc0360de2a None admin] PATH_INFO: `/v3/projects/be03e5300b3343169f35c643c93d1e06/users/5632ade0cc6f4ae28f72160de68486cb/roles/1e475e6a170a4dbabc4ce8348eeb30b9` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:19.619226 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:19.627874 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d42461-0926-45f7-9ad8-e5fc0360de2a None admin] RBAC: Authorizing `identity:create_grant(project_id=be03e5300b3343169f35c643c93d1e06, user_id=5632ade0cc6f4ae28f72160de68486cb, role_id=1e475e6a170a4dbabc4ce8348eeb30b9)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:19.629875 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d42461-0926-45f7-9ad8-e5fc0360de2a None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:19.654402 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7040036a-9765-4950-91d9-245d40b1e1d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:19.654402 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7040036a-9765-4950-91d9-245d40b1e1d5 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:19.654402 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7040036a-9765-4950-91d9-245d40b1e1d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:19.654402 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7040036a-9765-4950-91d9-245d40b1e1d5 None admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:19.654871 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7040036a-9765-4950-91d9-245d40b1e1d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:19.654871 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7040036a-9765-4950-91d9-245d40b1e1d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:19.654871 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7040036a-9765-4950-91d9-245d40b1e1d5 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:19.655382 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 308/602] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Sep 19 16:00:19 2022] PUT /identity/v3/projects/be03e5300b3343169f35c643c93d1e06/users/5632ade0cc6f4ae28f72160de68486cb/roles/1e475e6a170a4dbabc4ce8348eeb30b9 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:00:19.656968 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d42461-0926-45f7-9ad8-e5fc0360de2a None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:19.676825 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 295/603] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 19 16:00:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:19.685735 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7040036a-9765-4950-91d9-245d40b1e1d5 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:19.804643 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:19.854567 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d42461-0926-45f7-9ad8-e5fc0360de2a None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:19.895135 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7040036a-9765-4950-91d9-245d40b1e1d5 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:19.901922 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b403f64-e028-4f68-a744-004d03efa4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:19.902587 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b403f64-e028-4f68-a744-004d03efa4ce None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:19.902866 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b403f64-e028-4f68-a744-004d03efa4ce None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:19.903135 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b403f64-e028-4f68-a744-004d03efa4ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:19.913003 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b403f64-e028-4f68-a744-004d03efa4ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:19.914645 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b403f64-e028-4f68-a744-004d03efa4ce None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:19.927159 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-3b403f64-e028-4f68-a744-004d03efa4ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:00:19.927159 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:19.927159 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:19.927159 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:19.927159 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:19.927159 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:19.927159 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:19.927159 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:19.927159 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:19.927159 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:19.927159 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:19.927159 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:19.927159 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 16:00:19.927159 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 16:00:19.927159 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 16:00:19.928074 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 16:00:19.928074 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:19.928074 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:19.928074 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:19.928074 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:19.928074 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:19.928074 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:19.928074 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:19.928074 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:19.928074 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:19.928074 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:19.928074 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:19.928074 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:19.928074 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:19.928074 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:19.928074 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:19.928074 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:19.929661 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 16:00:19.929661 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 16:00:19.929661 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 16:00:19.929661 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 16:00:19.929661 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 16:00:19.929661 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 16:00:19.929661 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:00:19.929661 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:19.929661 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 309/604] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 19 16:00:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:19.929661 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514be076-6a03-4d70-839e-dec37228bc28 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:19.930183 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514be076-6a03-4d70-839e-dec37228bc28 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:19.930183 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514be076-6a03-4d70-839e-dec37228bc28 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:19.931591 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 310/605] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:00:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:00:19.937053 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514be076-6a03-4d70-839e-dec37228bc28 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:19.942506 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d980ff05-41c9-44a7-8eb4-c10bcfcf9443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:19.943422 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d980ff05-41c9-44a7-8eb4-c10bcfcf9443 None admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:19.943875 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d980ff05-41c9-44a7-8eb4-c10bcfcf9443 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:19.944319 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d980ff05-41c9-44a7-8eb4-c10bcfcf9443 None admin] PATH_INFO: `/v3/projects/be03e5300b3343169f35c643c93d1e06/users/7d596e43cc7446d29e2d62d9052d6091/roles/1e475e6a170a4dbabc4ce8348eeb30b9` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:19.965757 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d980ff05-41c9-44a7-8eb4-c10bcfcf9443 None admin] RBAC: Authorizing `identity:create_grant(project_id=be03e5300b3343169f35c643c93d1e06, user_id=7d596e43cc7446d29e2d62d9052d6091, role_id=1e475e6a170a4dbabc4ce8348eeb30b9)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:19.968237 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d980ff05-41c9-44a7-8eb4-c10bcfcf9443 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:19.994406 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 296/606] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Sep 19 16:00:19 2022] PUT /identity/v3/projects/be03e5300b3343169f35c643c93d1e06/users/7d596e43cc7446d29e2d62d9052d6091/roles/1e475e6a170a4dbabc4ce8348eeb30b9 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 16:00:19.994406 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b28ee2-d3ad-4620-b749-fe09bbffd232 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:19.994406 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b28ee2-d3ad-4620-b749-fe09bbffd232 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:19.994712 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b28ee2-d3ad-4620-b749-fe09bbffd232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:20.010082 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:20.032527 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-f8b28ee2-d3ad-4620-b749-fe09bbffd232 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:00:20.033770 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-f8b28ee2-d3ad-4620-b749-fe09bbffd232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:00:20.099783 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514be076-6a03-4d70-839e-dec37228bc28 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:20.174896 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a35912-8560-48f0-b2a0-d8cc7093a4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:20.176234 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a35912-8560-48f0-b2a0-d8cc7093a4b4 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:20.176525 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a35912-8560-48f0-b2a0-d8cc7093a4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:20.176801 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a35912-8560-48f0-b2a0-d8cc7093a4b4 None admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:20.177750 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a35912-8560-48f0-b2a0-d8cc7093a4b4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:20.178059 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a35912-8560-48f0-b2a0-d8cc7093a4b4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:20.179557 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a35912-8560-48f0-b2a0-d8cc7093a4b4 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:20.189811 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 297/607] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:00:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 16:00:20.208900 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 311/608] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 19 16:00:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:20.214098 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b28ee2-d3ad-4620-b749-fe09bbffd232 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:20.251476 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:20.348618 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b28ee2-d3ad-4620-b749-fe09bbffd232 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:20.398086 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51882b9e-f8eb-4378-8975-b27f05450730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:20.398810 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51882b9e-f8eb-4378-8975-b27f05450730 None admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:20.399092 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51882b9e-f8eb-4378-8975-b27f05450730 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:20.399370 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51882b9e-f8eb-4378-8975-b27f05450730 None admin] PATH_INFO: `/v3/projects/be03e5300b3343169f35c643c93d1e06/users/50355d7b1c6c4f0bbbbc7fe56efb9039/roles/1e475e6a170a4dbabc4ce8348eeb30b9` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:20.421862 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51882b9e-f8eb-4378-8975-b27f05450730 None admin] RBAC: Authorizing `identity:create_grant(project_id=be03e5300b3343169f35c643c93d1e06, user_id=50355d7b1c6c4f0bbbbc7fe56efb9039, role_id=1e475e6a170a4dbabc4ce8348eeb30b9)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:20.423440 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51882b9e-f8eb-4378-8975-b27f05450730 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:20.442535 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 298/609] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Sep 19 16:00:20 2022] PUT /identity/v3/projects/be03e5300b3343169f35c643c93d1e06/users/50355d7b1c6c4f0bbbbc7fe56efb9039/roles/1e475e6a170a4dbabc4ce8348eeb30b9 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:00:20.531773 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a35912-8560-48f0-b2a0-d8cc7093a4b4 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:20.539854 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a67bd2e-2cad-436c-8d26-f7259a106810 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:20.540170 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a67bd2e-2cad-436c-8d26-f7259a106810 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:20.540420 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a67bd2e-2cad-436c-8d26-f7259a106810 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:20.542144 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 299/610] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:00:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:00:20.548625 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e7111e-51a6-42cb-ad77-1e033e9a4fc1 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:20.548933 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e7111e-51a6-42cb-ad77-1e033e9a4fc1 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:20.549186 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e7111e-51a6-42cb-ad77-1e033e9a4fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:20.581497 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-69e7111e-51a6-42cb-ad77-1e033e9a4fc1 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:00:20.584242 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-69e7111e-51a6-42cb-ad77-1e033e9a4fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:00:20.706218 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a35912-8560-48f0-b2a0-d8cc7093a4b4 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:20.717479 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:20.718761 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 300/611] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:00:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 16:00:20.752633 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186616b5-91fa-4c8c-a639-62cb0221e8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:20.753263 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186616b5-91fa-4c8c-a639-62cb0221e8d7 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:20.753535 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186616b5-91fa-4c8c-a639-62cb0221e8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:20.753805 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186616b5-91fa-4c8c-a639-62cb0221e8d7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:20.771935 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186616b5-91fa-4c8c-a639-62cb0221e8d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 16:00:20.777717 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-186616b5-91fa-4c8c-a639-62cb0221e8d7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 16:00:20.777717 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:20.777717 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:20.777717 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:20.777717 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:20.777717 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:20.777717 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:20.777717 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:20.777717 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:20.777717 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:20.777717 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:20.777717 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:20.777717 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 16:00:20.777717 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 16:00:20.777717 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 16:00:20.777717 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 16:00:20.779270 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 16:00:20.779270 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 16:00:20.779270 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 16:00:20.779270 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 16:00:20.779270 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:20.779270 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:20.779270 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:20.779270 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:20.779270 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:20.779270 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:20.779270 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:20.779270 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:20.779270 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:20.779270 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:20.779270 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:20.779270 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:20.780118 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:20.780118 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:20.780118 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:20.780118 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:20.780118 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 16:00:20.780118 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 16:00:20.780118 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 16:00:20.780118 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 16:00:20.780118 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 16:00:20.780118 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 16:00:20.780118 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 16:00:20.780118 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:20.783389 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 312/612] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Sep 19 16:00:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:20.791940 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e7111e-51a6-42cb-ad77-1e033e9a4fc1 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:20.816849 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:20.916149 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e7111e-51a6-42cb-ad77-1e033e9a4fc1 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:20.955194 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ab5388-8a7c-44d6-83aa-150dee2aa8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:20.955810 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ab5388-8a7c-44d6-83aa-150dee2aa8f1 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:20.956093 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ab5388-8a7c-44d6-83aa-150dee2aa8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:20.956421 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ab5388-8a7c-44d6-83aa-150dee2aa8f1 None admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:20.957040 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ab5388-8a7c-44d6-83aa-150dee2aa8f1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:20.957342 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ab5388-8a7c-44d6-83aa-150dee2aa8f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:20.958761 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ab5388-8a7c-44d6-83aa-150dee2aa8f1 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:20.968200 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 301/613] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Sep 19 16:00:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:20.976577 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186616b5-91fa-4c8c-a639-62cb0221e8d7 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:21.074040 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ab5388-8a7c-44d6-83aa-150dee2aa8f1 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:21.192300 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186616b5-91fa-4c8c-a639-62cb0221e8d7 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:21.225363 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:21.253178 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb6d841-7f77-4391-a9cc-eb5b97bc9169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:21.253832 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb6d841-7f77-4391-a9cc-eb5b97bc9169 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:21.254113 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb6d841-7f77-4391-a9cc-eb5b97bc9169 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:21.254401 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb6d841-7f77-4391-a9cc-eb5b97bc9169 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:21.267551 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb6d841-7f77-4391-a9cc-eb5b97bc9169 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:21.269040 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb6d841-7f77-4391-a9cc-eb5b97bc9169 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:21.280550 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ab5388-8a7c-44d6-83aa-150dee2aa8f1 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:21.285366 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-cfb6d841-7f77-4391-a9cc-eb5b97bc9169 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:21.285366 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:21.285366 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:21.285366 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:21.285366 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:21.285366 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:21.285366 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:21.285366 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:21.285366 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:21.285366 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:21.285366 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:21.285366 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:21.285366 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:00:21.285366 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:00:21.285366 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:00:21.285366 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:00:21.286318 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:21.286318 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:21.286318 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:21.286318 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:21.286318 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:21.286318 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:21.286318 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:21.286318 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:21.286318 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:21.286318 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:21.286318 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:21.286318 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:21.286318 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:21.286318 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:21.286318 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:21.286318 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:21.286318 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:00:21.287231 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:00:21.287231 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:21.287231 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:21.287231 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 313/614] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:00:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:21.300801 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb6d841-7f77-4391-a9cc-eb5b97bc9169 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:21.328099 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1105198c-5ae6-420d-85eb-bce7ddcabc5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:21.328854 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1105198c-5ae6-420d-85eb-bce7ddcabc5f None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:21.329127 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1105198c-5ae6-420d-85eb-bce7ddcabc5f None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:21.329544 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1105198c-5ae6-420d-85eb-bce7ddcabc5f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:21.340274 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1105198c-5ae6-420d-85eb-bce7ddcabc5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 16:00:21.346544 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-1105198c-5ae6-420d-85eb-bce7ddcabc5f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 16:00:21.346544 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:21.346544 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:21.346544 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:21.346544 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:21.346544 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:21.346544 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:21.346544 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:21.346544 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:21.346544 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:21.346544 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:21.346544 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:21.346544 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 16:00:21.346544 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 16:00:21.346544 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 16:00:21.346544 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 16:00:21.347473 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 16:00:21.347473 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 16:00:21.347473 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 16:00:21.347473 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 16:00:21.347473 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:21.347473 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:21.347473 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:21.347473 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:21.347473 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:21.347473 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:21.347473 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:21.347473 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:21.347473 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:21.347473 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:21.347473 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:21.347473 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:21.348292 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:21.348292 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:21.348292 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:21.348292 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:21.348292 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 16:00:21.348292 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 16:00:21.348292 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 16:00:21.348292 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 16:00:21.348292 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 16:00:21.348292 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 16:00:21.348292 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 16:00:21.348292 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:21.352510 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 302/615] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Sep 19 16:00:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:21.353698 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1105198c-5ae6-420d-85eb-bce7ddcabc5f None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:21.369790 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:21.502252 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb6d841-7f77-4391-a9cc-eb5b97bc9169 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:21.516502 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1105198c-5ae6-420d-85eb-bce7ddcabc5f None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:21.566268 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a84b136-fc27-409f-aef0-d5146527bd71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:21.566974 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a84b136-fc27-409f-aef0-d5146527bd71 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:21.567172 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a84b136-fc27-409f-aef0-d5146527bd71 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:21.567446 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a84b136-fc27-409f-aef0-d5146527bd71 None admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:21.568094 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a84b136-fc27-409f-aef0-d5146527bd71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:21.568398 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a84b136-fc27-409f-aef0-d5146527bd71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:21.569851 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a84b136-fc27-409f-aef0-d5146527bd71 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:21.578004 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee573e44-c90f-4622-938b-38b147581a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:21.578655 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee573e44-c90f-4622-938b-38b147581a0a None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:21.578935 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee573e44-c90f-4622-938b-38b147581a0a None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:21.579206 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee573e44-c90f-4622-938b-38b147581a0a None admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:21.579866 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee573e44-c90f-4622-938b-38b147581a0a None admin] RBAC: Adding query filter params (name=service) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:21.580167 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee573e44-c90f-4622-938b-38b147581a0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:21.581528 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee573e44-c90f-4622-938b-38b147581a0a None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:21.588207 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:21.596105 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 314/616] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:00:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:00:21.598110 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 303/617] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Sep 19 16:00:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:21.605247 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a84b136-fc27-409f-aef0-d5146527bd71 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:21.608819 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee573e44-c90f-4622-938b-38b147581a0a None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:21.754880 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:21.769411 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a84b136-fc27-409f-aef0-d5146527bd71 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:21.787449 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee573e44-c90f-4622-938b-38b147581a0a None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:21.822029 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a0073d-0ce6-4252-bde6-385d95b131bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:21.822707 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a0073d-0ce6-4252-bde6-385d95b131bb None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:21.822992 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a0073d-0ce6-4252-bde6-385d95b131bb None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:21.823269 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a0073d-0ce6-4252-bde6-385d95b131bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:21.834403 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a0073d-0ce6-4252-bde6-385d95b131bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:21.835870 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a0073d-0ce6-4252-bde6-385d95b131bb None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:21.851167 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-e1a0073d-0ce6-4252-bde6-385d95b131bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:21.851167 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:21.851167 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:21.851167 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:21.851167 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:21.851167 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:21.851167 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:21.851167 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:21.851167 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:21.851167 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:21.851167 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:21.851167 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:21.851167 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:00:21.851167 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:00:21.851167 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:00:21.851167 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:00:21.852124 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:21.852124 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:21.852124 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:21.852124 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:21.852124 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:21.852124 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:21.852124 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:21.852124 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:21.852124 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:21.852124 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:21.852124 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:21.852124 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:21.852124 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:21.852124 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:21.852124 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:21.852124 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:21.852124 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:00:21.853083 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:00:21.853083 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:21.853083 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:21.853083 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 304/618] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:00:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:21.855257 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1b6814-294d-4911-9401-37f67855214d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:21.855994 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b6814-294d-4911-9401-37f67855214d None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:21.856270 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b6814-294d-4911-9401-37f67855214d None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:21.856596 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b6814-294d-4911-9401-37f67855214d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:21.863221 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a0073d-0ce6-4252-bde6-385d95b131bb None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:21.868177 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1b6814-294d-4911-9401-37f67855214d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:21.869625 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1b6814-294d-4911-9401-37f67855214d None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:21.882213 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-1d1b6814-294d-4911-9401-37f67855214d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 19 16:00:21.882213 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:21.882213 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:21.882213 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:21.882213 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:21.882213 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:21.882213 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:21.882213 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:21.882213 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:21.882213 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:21.882213 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:21.882213 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:21.882213 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 16:00:21.882213 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 16:00:21.882213 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 16:00:21.882213 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 16:00:21.883394 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:21.883394 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:21.883394 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 16:00:21.883394 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:00:21.883394 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 16:00:21.883394 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:00:21.883394 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:21.883394 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:21.883394 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:21.883394 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:21.883394 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:21.883394 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:21.883394 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:21.883394 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:21.883394 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:21.883394 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:21.883394 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:21.883394 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:21.885655 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:21.885655 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:21.885655 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 16:00:21.885655 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 16:00:21.885655 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 16:00:21.885655 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 16:00:21.885655 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 16:00:21.885655 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 16:00:21.885655 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 19 16:00:21.885655 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:21.888570 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 315/619] 10.222.0.57 () {56 vars in 1093 bytes} [Mon Sep 19 16:00:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 16:00:21.889783 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1b6814-294d-4911-9401-37f67855214d None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:21.942781 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:22.026185 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a0073d-0ce6-4252-bde6-385d95b131bb None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:22.075337 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1b6814-294d-4911-9401-37f67855214d None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:22.093555 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3f0fc0-4fed-43ff-b4f5-456bd3457b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:22.094226 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3f0fc0-4fed-43ff-b4f5-456bd3457b08 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:22.094537 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3f0fc0-4fed-43ff-b4f5-456bd3457b08 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:22.094812 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3f0fc0-4fed-43ff-b4f5-456bd3457b08 None admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:22.095458 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3f0fc0-4fed-43ff-b4f5-456bd3457b08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:22.095760 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3f0fc0-4fed-43ff-b4f5-456bd3457b08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:22.097127 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3f0fc0-4fed-43ff-b4f5-456bd3457b08 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:22.126841 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 305/620] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:00:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:00:22.130103 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3f0fc0-4fed-43ff-b4f5-456bd3457b08 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:22.148277 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16c6ba6-ae37-456f-a4fd-cd9cae9313ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:22.148907 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16c6ba6-ae37-456f-a4fd-cd9cae9313ab None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:22.149181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16c6ba6-ae37-456f-a4fd-cd9cae9313ab None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:22.149452 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16c6ba6-ae37-456f-a4fd-cd9cae9313ab None admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:22.150200 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16c6ba6-ae37-456f-a4fd-cd9cae9313ab None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:22.150529 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16c6ba6-ae37-456f-a4fd-cd9cae9313ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:22.151993 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16c6ba6-ae37-456f-a4fd-cd9cae9313ab None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:22.165770 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:22.181302 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 316/621] 10.222.0.57 () {56 vars in 1133 bytes} [Mon Sep 19 16:00:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:00:22.182955 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16c6ba6-ae37-456f-a4fd-cd9cae9313ab None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:22.318206 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3f0fc0-4fed-43ff-b4f5-456bd3457b08 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:22.321422 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16c6ba6-ae37-456f-a4fd-cd9cae9313ab None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:22.330846 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:22.358310 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6446ed7-8cd4-475a-adc5-11593ab46fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:22.358966 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6446ed7-8cd4-475a-adc5-11593ab46fc2 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:22.359247 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6446ed7-8cd4-475a-adc5-11593ab46fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:22.359523 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6446ed7-8cd4-475a-adc5-11593ab46fc2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:22.360474 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0a7097-9c80-451b-8976-679bc296646b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:22.361224 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0a7097-9c80-451b-8976-679bc296646b None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:22.361598 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0a7097-9c80-451b-8976-679bc296646b None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:22.361964 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0a7097-9c80-451b-8976-679bc296646b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:22.370112 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0a7097-9c80-451b-8976-679bc296646b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:22.371811 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0a7097-9c80-451b-8976-679bc296646b None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:22.386142 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-9c0a7097-9c80-451b-8976-679bc296646b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 19 16:00:22.386142 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:22.386142 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:22.386142 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:22.386142 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:22.386142 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:22.386142 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:22.386142 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:22.386142 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:22.386142 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:22.386142 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:22.386142 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:22.386142 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 16:00:22.386142 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 16:00:22.386142 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 16:00:22.386142 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 16:00:22.387163 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:22.387163 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:22.387163 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 16:00:22.387163 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:00:22.387163 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 16:00:22.387163 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:00:22.387163 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:22.387163 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:22.387163 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:22.387163 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:22.387163 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:22.387163 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:22.387163 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:22.387163 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:22.387163 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:22.387163 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:22.387163 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:22.387163 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:22.388683 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:22.388683 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:22.388683 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 16:00:22.388683 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 16:00:22.388683 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 16:00:22.388683 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 16:00:22.388683 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 16:00:22.388683 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 16:00:22.388683 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 19 16:00:22.388683 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:22.390216 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 306/622] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 19 16:00:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 16:00:22.390362 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa00027-44fc-4f7c-8cd5-6f87c49f532f None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:22.391534 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa00027-44fc-4f7c-8cd5-6f87c49f532f None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:22.391534 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa00027-44fc-4f7c-8cd5-6f87c49f532f None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:22.394404 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 307/623] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:00:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:00:22.399338 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa00027-44fc-4f7c-8cd5-6f87c49f532f None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:22.433162 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:22.437476 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6446ed7-8cd4-475a-adc5-11593ab46fc2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:22.439346 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6446ed7-8cd4-475a-adc5-11593ab46fc2 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:22.527931 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa00027-44fc-4f7c-8cd5-6f87c49f532f None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:22.547843 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 317/624] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 19 16:00:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 16:00:22.549174 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6e0a88-636a-4715-ac07-f07133b3b1b9 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:22.549456 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6e0a88-636a-4715-ac07-f07133b3b1b9 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:22.549730 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6e0a88-636a-4715-ac07-f07133b3b1b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:22.569643 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-ab6e0a88-636a-4715-ac07-f07133b3b1b9 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:00:22.570805 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-ab6e0a88-636a-4715-ac07-f07133b3b1b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:00:22.571751 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c6d919-d745-4c7f-9b4a-fa522c8cb947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:22.572374 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c6d919-d745-4c7f-9b4a-fa522c8cb947 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:22.572655 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c6d919-d745-4c7f-9b4a-fa522c8cb947 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:22.572936 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c6d919-d745-4c7f-9b4a-fa522c8cb947 None admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:22.573663 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c6d919-d745-4c7f-9b4a-fa522c8cb947 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:22.573971 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c6d919-d745-4c7f-9b4a-fa522c8cb947 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:22.575952 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c6d919-d745-4c7f-9b4a-fa522c8cb947 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:22.594994 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 308/625] 10.222.0.57 () {56 vars in 1135 bytes} [Mon Sep 19 16:00:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:22.596122 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c6d919-d745-4c7f-9b4a-fa522c8cb947 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:22.633263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:22.662669 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 318/626] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:00:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 16:00:22.663916 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6e0a88-636a-4715-ac07-f07133b3b1b9 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:22.731884 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c6d919-d745-4c7f-9b4a-fa522c8cb947 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:22.739740 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:22.769152 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f533b7d-fe6f-4cab-9dee-30087aafc30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:22.769797 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f533b7d-fe6f-4cab-9dee-30087aafc30f None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:22.770077 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f533b7d-fe6f-4cab-9dee-30087aafc30f None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:22.770355 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f533b7d-fe6f-4cab-9dee-30087aafc30f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:22.781510 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f533b7d-fe6f-4cab-9dee-30087aafc30f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:22.783364 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f533b7d-fe6f-4cab-9dee-30087aafc30f None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:22.800494 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-0f533b7d-fe6f-4cab-9dee-30087aafc30f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:22.800494 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:22.800494 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:22.800494 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:22.800494 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:22.800494 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:22.800494 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:22.800494 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:22.800494 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:22.800494 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:22.800494 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:22.800494 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:22.800494 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:00:22.800494 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:00:22.800494 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:00:22.800494 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:00:22.801710 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:22.801710 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:22.801710 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:22.801710 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:22.801710 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:22.801710 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:22.801710 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:22.801710 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:22.801710 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:22.801710 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:22.801710 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:22.801710 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:22.801710 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:22.801710 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:22.801710 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:22.801710 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:22.801710 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:00:22.802753 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:00:22.802753 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:22.802753 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:22.802753 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 309/627] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:00:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:22.811462 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f533b7d-fe6f-4cab-9dee-30087aafc30f None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:22.906765 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6e0a88-636a-4715-ac07-f07133b3b1b9 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:22.907438 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:22.928971 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f533b7d-fe6f-4cab-9dee-30087aafc30f None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:22.951948 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b902135-3440-4250-887a-7e81b91c50a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:22.952646 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b902135-3440-4250-887a-7e81b91c50a8 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:22.953061 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b902135-3440-4250-887a-7e81b91c50a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:22.953493 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b902135-3440-4250-887a-7e81b91c50a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:22.966209 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f39ee5b-43d7-464b-9dcf-974b1f41aefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:22.966990 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f39ee5b-43d7-464b-9dcf-974b1f41aefe None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:22.967205 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b902135-3440-4250-887a-7e81b91c50a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:22.967446 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f39ee5b-43d7-464b-9dcf-974b1f41aefe None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:22.967811 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f39ee5b-43d7-464b-9dcf-974b1f41aefe None admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:22.968551 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f39ee5b-43d7-464b-9dcf-974b1f41aefe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:22.968664 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b902135-3440-4250-887a-7e81b91c50a8 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:22.969024 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f39ee5b-43d7-464b-9dcf-974b1f41aefe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:22.970473 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f39ee5b-43d7-464b-9dcf-974b1f41aefe None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:22.985318 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 310/628] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:00:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:22.986315 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-3b902135-3440-4250-887a-7e81b91c50a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:22.986315 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:22.986315 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:22.986315 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:22.986315 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:22.986315 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:22.986315 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:22.986315 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:22.986315 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:22.986315 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:22.986315 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:22.986315 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:22.986315 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:00:22.986315 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:00:22.986315 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:00:22.986315 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:00:22.987224 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:22.987224 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:22.987224 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:22.987224 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:22.987224 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:22.987224 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:22.987224 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:22.987224 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:22.987224 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:22.987224 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:22.987224 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:22.987224 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:22.987224 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:22.987224 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:22.987224 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:22.987224 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:22.987224 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:00:22.988105 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:00:22.988105 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:22.988105 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:22.988270 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41a1356-dc2a-4632-8d0a-183e47773cfe None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:22.988270 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41a1356-dc2a-4632-8d0a-183e47773cfe None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:22.988270 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41a1356-dc2a-4632-8d0a-183e47773cfe None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:22.988455 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 319/629] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:00:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:22.988521 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b902135-3440-4250-887a-7e81b91c50a8 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:22.992498 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 311/630] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:00:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:00:22.993594 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41a1356-dc2a-4632-8d0a-183e47773cfe None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:23.087661 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:23.109604 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b902135-3440-4250-887a-7e81b91c50a8 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:23.115562 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41a1356-dc2a-4632-8d0a-183e47773cfe None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:23.148623 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172142fb-c272-4836-86da-2cbf377f0392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:23.149260 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172142fb-c272-4836-86da-2cbf377f0392 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:23.149530 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172142fb-c272-4836-86da-2cbf377f0392 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:23.149800 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172142fb-c272-4836-86da-2cbf377f0392 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:23.151459 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744f31cd-79b3-4dad-8896-ead4fff75bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:23.152072 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f31cd-79b3-4dad-8896-ead4fff75bbf None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:23.152351 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f31cd-79b3-4dad-8896-ead4fff75bbf None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:23.152625 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f31cd-79b3-4dad-8896-ead4fff75bbf None admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:23.153752 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744f31cd-79b3-4dad-8896-ead4fff75bbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:23.154053 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744f31cd-79b3-4dad-8896-ead4fff75bbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:23.155551 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744f31cd-79b3-4dad-8896-ead4fff75bbf None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:23.161632 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172142fb-c272-4836-86da-2cbf377f0392 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 16:00:23.165266 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:23.167628 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-172142fb-c272-4836-86da-2cbf377f0392 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 16:00:23.167628 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:23.167628 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:23.167628 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:23.167628 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:23.167628 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:23.167628 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:23.167628 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:23.167628 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:23.167628 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:23.167628 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:23.167628 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:23.167628 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 16:00:23.167628 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 16:00:23.167628 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 16:00:23.167628 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 16:00:23.168803 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 16:00:23.168803 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 16:00:23.168803 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 16:00:23.168803 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 16:00:23.168803 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:23.168803 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:23.168803 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:23.168803 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:23.168803 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:23.168803 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:23.168803 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:23.168803 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:23.168803 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:23.168803 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:23.168803 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:23.168803 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:23.175660 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:23.175660 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:23.175660 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:23.175660 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:23.175660 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 16:00:23.175660 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 16:00:23.175660 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 16:00:23.175660 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 16:00:23.175660 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 16:00:23.175660 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 16:00:23.175660 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 16:00:23.175660 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:23.176372 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 312/632] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:00:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:23.176445 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 320/632] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Sep 19 16:00:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:23.176445 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172142fb-c272-4836-86da-2cbf377f0392 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:23.176571 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2780767-3ac0-45dd-b6ce-97534e0c2c86 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:23.176571 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2780767-3ac0-45dd-b6ce-97534e0c2c86 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:23.176571 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2780767-3ac0-45dd-b6ce-97534e0c2c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:23.193041 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-f2780767-3ac0-45dd-b6ce-97534e0c2c86 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:00:23.194237 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-f2780767-3ac0-45dd-b6ce-97534e0c2c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:00:23.259264 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 313/633] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:00:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 16:00:23.260100 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2780767-3ac0-45dd-b6ce-97534e0c2c86 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:23.273256 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172142fb-c272-4836-86da-2cbf377f0392 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:23.314294 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6363502f-55c2-4bbc-ad6e-c158649d9e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:23.314957 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6363502f-55c2-4bbc-ad6e-c158649d9e57 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:23.315234 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6363502f-55c2-4bbc-ad6e-c158649d9e57 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:23.315502 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6363502f-55c2-4bbc-ad6e-c158649d9e57 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:23.322725 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6363502f-55c2-4bbc-ad6e-c158649d9e57 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:23.324155 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6363502f-55c2-4bbc-ad6e-c158649d9e57 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:23.327813 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:23.338825 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-6363502f-55c2-4bbc-ad6e-c158649d9e57 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:00:23.338825 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:23.338825 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:23.338825 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:23.338825 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:23.338825 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:23.338825 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:23.338825 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:23.338825 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:23.338825 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:23.338825 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:23.338825 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:23.338825 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 16:00:23.338825 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 16:00:23.338825 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 16:00:23.339958 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 16:00:23.339958 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:23.339958 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:23.339958 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:23.339958 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:23.339958 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:23.339958 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:23.339958 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:23.339958 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:23.339958 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:23.339958 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:23.339958 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:23.339958 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:23.339958 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:23.339958 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:23.339958 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:23.339958 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:23.341054 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 16:00:23.341054 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 16:00:23.341054 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 16:00:23.341054 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 16:00:23.341054 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 16:00:23.341054 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 16:00:23.341054 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:00:23.341054 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:23.343836 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 321/634] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 19 16:00:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:23.345110 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e7c2a1-a34c-4342-afc5-2cdf481623e5 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:23.345392 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e7c2a1-a34c-4342-afc5-2cdf481623e5 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:23.345665 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e7c2a1-a34c-4342-afc5-2cdf481623e5 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:23.350717 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 322/635] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:00:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:00:23.351582 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e7c2a1-a34c-4342-afc5-2cdf481623e5 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:23.385130 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2780767-3ac0-45dd-b6ce-97534e0c2c86 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:23.416973 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:23.424503 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b6322-d344-41f4-ad53-525be1d65510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:23.425127 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b6322-d344-41f4-ad53-525be1d65510 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:23.425409 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b6322-d344-41f4-ad53-525be1d65510 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:23.425681 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b6322-d344-41f4-ad53-525be1d65510 None admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:23.426350 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186b6322-d344-41f4-ad53-525be1d65510 None admin] RBAC: Adding query filter params (name=service) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:23.426687 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186b6322-d344-41f4-ad53-525be1d65510 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:23.428051 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186b6322-d344-41f4-ad53-525be1d65510 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:23.438079 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 314/636] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Sep 19 16:00:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:23.439211 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b6322-d344-41f4-ad53-525be1d65510 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:23.468131 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e7c2a1-a34c-4342-afc5-2cdf481623e5 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:23.507583 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7a1522-bab0-4409-9f24-53f4f3b95d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:23.508631 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7a1522-bab0-4409-9f24-53f4f3b95d00 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:23.509151 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7a1522-bab0-4409-9f24-53f4f3b95d00 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:23.509491 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7a1522-bab0-4409-9f24-53f4f3b95d00 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:23.516666 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7a1522-bab0-4409-9f24-53f4f3b95d00 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:23.518202 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7a1522-bab0-4409-9f24-53f4f3b95d00 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:23.530334 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-5e7a1522-bab0-4409-9f24-53f4f3b95d00 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:00:23.530334 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:23.530334 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:23.530334 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:23.530334 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:23.530334 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:23.530334 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:23.530334 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:23.530334 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:23.530334 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:23.530334 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:23.530334 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:23.530334 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 16:00:23.530334 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 16:00:23.530334 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 16:00:23.531492 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 16:00:23.531492 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:23.531492 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:23.531492 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:23.531492 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:23.531492 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:23.531492 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:23.531492 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:23.531492 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:23.531492 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:23.531492 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:23.531492 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:23.531492 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:23.531492 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:23.531492 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:23.531492 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:23.531492 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:23.532589 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 16:00:23.532589 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 16:00:23.532589 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 16:00:23.532589 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 16:00:23.532589 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 16:00:23.532589 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 16:00:23.532589 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:00:23.532589 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:23.532589 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 323/637] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 19 16:00:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:23.532589 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff5e5d4-41b5-49da-8b43-0a817218c328 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:23.533245 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff5e5d4-41b5-49da-8b43-0a817218c328 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:23.533245 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff5e5d4-41b5-49da-8b43-0a817218c328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:23.552849 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-aff5e5d4-41b5-49da-8b43-0a817218c328 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:00:23.553200 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b6322-d344-41f4-ad53-525be1d65510 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:23.554045 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-aff5e5d4-41b5-49da-8b43-0a817218c328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:00:23.588345 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9660522a-8831-4984-ac17-9727e75f4f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:23.588950 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9660522a-8831-4984-ac17-9727e75f4f1c None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:23.589229 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9660522a-8831-4984-ac17-9727e75f4f1c None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:23.589503 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9660522a-8831-4984-ac17-9727e75f4f1c None admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:23.590194 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9660522a-8831-4984-ac17-9727e75f4f1c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:23.590535 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9660522a-8831-4984-ac17-9727e75f4f1c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:23.591989 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9660522a-8831-4984-ac17-9727e75f4f1c None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:23.604185 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 315/638] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 19 16:00:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:23.605023 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9660522a-8831-4984-ac17-9727e75f4f1c None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:23.611507 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:23.625441 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 324/639] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:00:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 16:00:23.626366 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff5e5d4-41b5-49da-8b43-0a817218c328 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:23.663111 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:23.724340 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9660522a-8831-4984-ac17-9727e75f4f1c None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:23.754628 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff5e5d4-41b5-49da-8b43-0a817218c328 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:23.772541 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d87dd4-99f1-4512-894c-29e2868b472c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:23.773647 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d87dd4-99f1-4512-894c-29e2868b472c None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:23.773922 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d87dd4-99f1-4512-894c-29e2868b472c None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:23.774217 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d87dd4-99f1-4512-894c-29e2868b472c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:23.787144 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d87dd4-99f1-4512-894c-29e2868b472c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:23.788893 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d87dd4-99f1-4512-894c-29e2868b472c None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:23.798362 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628492df-bd2d-421a-b20c-f1c729c271e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:23.799036 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628492df-bd2d-421a-b20c-f1c729c271e4 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:23.799316 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628492df-bd2d-421a-b20c-f1c729c271e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:23.799591 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628492df-bd2d-421a-b20c-f1c729c271e4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:23.807280 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-47d87dd4-99f1-4512-894c-29e2868b472c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:23.807280 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:23.807280 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:23.807280 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:23.807280 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:23.807280 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:23.807280 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:23.807280 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:23.807280 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:23.807280 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:23.807280 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:23.807280 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:23.807280 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:00:23.807280 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:00:23.807280 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:00:23.807280 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:00:23.808495 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:23.808495 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:23.808495 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:23.808495 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:23.808495 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:23.808495 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:23.808495 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:23.808495 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:23.808495 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:23.808495 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:23.808495 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:23.808495 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:23.808495 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:23.808495 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:23.808495 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:23.808495 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:23.808495 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:00:23.809653 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:00:23.809653 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:23.809653 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:23.812024 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 316/640] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:00:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:23.813036 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d87dd4-99f1-4512-894c-29e2868b472c None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:23.814477 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628492df-bd2d-421a-b20c-f1c729c271e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 16:00:23.819878 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-628492df-bd2d-421a-b20c-f1c729c271e4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 16:00:23.819878 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:23.819878 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:23.819878 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:23.819878 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:23.819878 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:23.819878 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:23.819878 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:23.819878 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:23.819878 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:23.819878 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:23.819878 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:23.819878 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 16:00:23.819878 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 16:00:23.819878 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 16:00:23.819878 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 16:00:23.820901 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 16:00:23.820901 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 16:00:23.820901 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 16:00:23.820901 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 16:00:23.820901 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:23.820901 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:23.820901 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:23.820901 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:23.820901 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:23.820901 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:23.820901 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:23.820901 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:23.820901 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:23.820901 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:23.820901 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:23.820901 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:23.821740 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:23.821740 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:23.821740 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:23.821740 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:23.821740 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 16:00:23.821740 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 16:00:23.821740 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 16:00:23.821740 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 16:00:23.821740 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 16:00:23.821740 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 16:00:23.821740 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 16:00:23.821740 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:23.821740 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 325/641] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Sep 19 16:00:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:23.822458 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628492df-bd2d-421a-b20c-f1c729c271e4 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:23.881846 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:23.937540 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628492df-bd2d-421a-b20c-f1c729c271e4 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:23.940376 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d87dd4-99f1-4512-894c-29e2868b472c None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:23.942775 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:23.973471 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9193090-4772-4808-baca-bcde6372b917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:23.974085 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9193090-4772-4808-baca-bcde6372b917 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:23.974425 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9193090-4772-4808-baca-bcde6372b917 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:23.974662 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9193090-4772-4808-baca-bcde6372b917 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:23.974769 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16627ea0-e80f-4794-9a9e-7ae05f16d5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:23.975313 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16627ea0-e80f-4794-9a9e-7ae05f16d5a3 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:23.975411 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9193090-4772-4808-baca-bcde6372b917 None admin] RBAC: Adding query filter params (role.id=1e475e6a170a4dbabc4ce8348eeb30b9, scope.project.id=be03e5300b3343169f35c643c93d1e06, user.id=1ecd01cc72834b72b5ff2c4b47249737) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:23.975594 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16627ea0-e80f-4794-9a9e-7ae05f16d5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:23.975691 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9193090-4772-4808-baca-bcde6372b917 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:23.975867 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16627ea0-e80f-4794-9a9e-7ae05f16d5a3 None admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:23.976550 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16627ea0-e80f-4794-9a9e-7ae05f16d5a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:23.976857 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16627ea0-e80f-4794-9a9e-7ae05f16d5a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:23.977098 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9193090-4772-4808-baca-bcde6372b917 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:23.978255 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16627ea0-e80f-4794-9a9e-7ae05f16d5a3 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:23.984269 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 326/642] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 19 16:00:23 2022] GET /identity/v3/role_assignments?user.id=1ecd01cc72834b72b5ff2c4b47249737&scope.project.id=be03e5300b3343169f35c643c93d1e06&role.id=1e475e6a170a4dbabc4ce8348eeb30b9 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:23.984855 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9193090-4772-4808-baca-bcde6372b917 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:23.992429 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 317/643] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 19 16:00:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:23.993314 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16627ea0-e80f-4794-9a9e-7ae05f16d5a3 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:24.110485 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9193090-4772-4808-baca-bcde6372b917 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:24.117013 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16627ea0-e80f-4794-9a9e-7ae05f16d5a3 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:24.128269 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:24.154475 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3796a33-049a-4af2-8e08-36875177e79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:24.155024 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d781834b-c258-4dd2-91af-51edc3fab4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:24.155142 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3796a33-049a-4af2-8e08-36875177e79c None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:24.155353 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3796a33-049a-4af2-8e08-36875177e79c None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:24.155657 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d781834b-c258-4dd2-91af-51edc3fab4d6 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:24.155936 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d781834b-c258-4dd2-91af-51edc3fab4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:24.156208 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d781834b-c258-4dd2-91af-51edc3fab4d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:24.156311 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3796a33-049a-4af2-8e08-36875177e79c None admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:24.156856 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d781834b-c258-4dd2-91af-51edc3fab4d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:24.156953 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3796a33-049a-4af2-8e08-36875177e79c None admin] RBAC: Adding query filter params (name=service) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:24.157158 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d781834b-c258-4dd2-91af-51edc3fab4d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:24.157253 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3796a33-049a-4af2-8e08-36875177e79c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:24.158552 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d781834b-c258-4dd2-91af-51edc3fab4d6 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:24.158663 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3796a33-049a-4af2-8e08-36875177e79c None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:24.171843 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 318/644] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Sep 19 16:00:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:24.171843 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3796a33-049a-4af2-8e08-36875177e79c None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:24.172110 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 327/645] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:00:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:24.172403 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d781834b-c258-4dd2-91af-51edc3fab4d6 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:24.204250 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:24.293607 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3796a33-049a-4af2-8e08-36875177e79c None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:24.300142 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d781834b-c258-4dd2-91af-51edc3fab4d6 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:24.332422 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a4f812-f62b-4cf2-807a-c83650646f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:24.333057 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a4f812-f62b-4cf2-807a-c83650646f69 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:24.333340 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a4f812-f62b-4cf2-807a-c83650646f69 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:24.333616 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a4f812-f62b-4cf2-807a-c83650646f69 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:24.335291 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c659c41-0232-40ad-ab39-e77d66fa0530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:24.335949 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c659c41-0232-40ad-ab39-e77d66fa0530 None admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:24.336227 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c659c41-0232-40ad-ab39-e77d66fa0530 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:24.336504 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c659c41-0232-40ad-ab39-e77d66fa0530 None admin] PATH_INFO: `/v3/projects/be03e5300b3343169f35c643c93d1e06/users/884a0524aca744b99c7496c11c3291ae/roles/1e475e6a170a4dbabc4ce8348eeb30b9` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:24.345586 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a4f812-f62b-4cf2-807a-c83650646f69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 16:00:24.351115 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-13a4f812-f62b-4cf2-807a-c83650646f69 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 16:00:24.351115 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:24.351115 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:24.351115 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:24.351115 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:24.351115 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:24.351115 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:24.351115 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:24.351115 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:24.351115 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:24.351115 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:24.351115 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:24.351115 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 16:00:24.351115 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 16:00:24.351115 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 16:00:24.351115 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 16:00:24.352081 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 16:00:24.352081 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 16:00:24.352081 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 16:00:24.352081 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 16:00:24.352081 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:24.352081 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:24.352081 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:24.352081 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:24.352081 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:24.352081 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:24.352081 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:24.352081 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:24.352081 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:24.352081 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:24.352081 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:24.352081 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:24.353055 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:24.353055 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:24.353055 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:24.353055 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:24.353055 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 16:00:24.353055 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 16:00:24.353055 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 16:00:24.353055 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 16:00:24.353055 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 16:00:24.353055 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 16:00:24.353055 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 16:00:24.353055 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:24.353055 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 319/646] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Sep 19 16:00:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:24.353055 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a4f812-f62b-4cf2-807a-c83650646f69 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:24.354967 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c659c41-0232-40ad-ab39-e77d66fa0530 None admin] RBAC: Authorizing `identity:create_grant(project_id=be03e5300b3343169f35c643c93d1e06, user_id=884a0524aca744b99c7496c11c3291ae, role_id=1e475e6a170a4dbabc4ce8348eeb30b9)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:24.356469 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c659c41-0232-40ad-ab39-e77d66fa0530 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:24.376399 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 328/647] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Sep 19 16:00:24 2022] PUT /identity/v3/projects/be03e5300b3343169f35c643c93d1e06/users/884a0524aca744b99c7496c11c3291ae/roles/1e475e6a170a4dbabc4ce8348eeb30b9 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:00:24.376399 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c659c41-0232-40ad-ab39-e77d66fa0530 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:24.416715 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:24.477747 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:24.502883 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a4f812-f62b-4cf2-807a-c83650646f69 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:24.535266 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c659c41-0232-40ad-ab39-e77d66fa0530 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:24.547718 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713f70d6-b030-44c1-b450-9f0daa9dd5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:24.548715 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f70d6-b030-44c1-b450-9f0daa9dd5cc None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:24.549215 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f70d6-b030-44c1-b450-9f0daa9dd5cc None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:24.549697 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f70d6-b030-44c1-b450-9f0daa9dd5cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:24.562018 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713f70d6-b030-44c1-b450-9f0daa9dd5cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:24.563974 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713f70d6-b030-44c1-b450-9f0daa9dd5cc None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:24.580781 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eebdcd3-5d03-44d9-9ce6-a9b59ef1705e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:24.581440 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eebdcd3-5d03-44d9-9ce6-a9b59ef1705e None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:24.581719 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eebdcd3-5d03-44d9-9ce6-a9b59ef1705e None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:24.581993 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eebdcd3-5d03-44d9-9ce6-a9b59ef1705e None admin] PATH_INFO: `/v3/users/glance` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:24.587047 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-713f70d6-b030-44c1-b450-9f0daa9dd5cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:24.587047 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:24.587047 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:24.587047 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:24.587047 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:24.587047 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:24.587047 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:24.587047 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:24.587047 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:24.587047 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:24.587047 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:24.587047 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:24.587047 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:00:24.587047 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:00:24.587047 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:00:24.587047 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:00:24.588012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:24.588012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:24.588012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:24.588012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:24.588012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:24.588012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:24.588012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:24.588012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:24.588012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:24.588012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:24.588012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:24.588012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:24.588012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:24.588012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:24.588012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:24.588012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:24.588012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:00:24.588907 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:00:24.588907 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:24.588907 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:24.588907 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 320/648] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:00:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:24.589334 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713f70d6-b030-44c1-b450-9f0daa9dd5cc None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:24.591639 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eebdcd3-5d03-44d9-9ce6-a9b59ef1705e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:24.592670 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eebdcd3-5d03-44d9-9ce6-a9b59ef1705e None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:24.605130 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-7eebdcd3-5d03-44d9-9ce6-a9b59ef1705e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 19 16:00:24.605130 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:24.605130 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:24.605130 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:24.605130 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:24.605130 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:24.605130 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:24.605130 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:24.605130 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:24.605130 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:24.605130 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:24.605130 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:24.605130 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 16:00:24.605130 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 16:00:24.605130 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 16:00:24.605130 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 16:00:24.606119 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:24.606119 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:24.606119 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 16:00:24.606119 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:00:24.606119 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 16:00:24.606119 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:00:24.606119 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:24.606119 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:24.606119 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:24.606119 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:24.606119 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:24.606119 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:24.606119 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:24.606119 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:24.606119 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:24.606119 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:24.606119 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:24.606119 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:24.607092 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:24.607092 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:24.607092 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 16:00:24.607092 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 16:00:24.607092 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 16:00:24.607092 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 16:00:24.607092 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 16:00:24.607092 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 16:00:24.607092 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 19 16:00:24.607092 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:24.607627 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 329/649] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 16:00:24 2022] GET /identity/v3/users/glance => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:24.608384 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eebdcd3-5d03-44d9-9ce6-a9b59ef1705e None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:24.723134 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eebdcd3-5d03-44d9-9ce6-a9b59ef1705e None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:24.725065 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713f70d6-b030-44c1-b450-9f0daa9dd5cc None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:24.726714 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:24.773162 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2cbb62-cbf6-4e61-98d1-b6ad25225fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:24.773780 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2cbb62-cbf6-4e61-98d1-b6ad25225fd8 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:24.774058 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2cbb62-cbf6-4e61-98d1-b6ad25225fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:24.774331 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2cbb62-cbf6-4e61-98d1-b6ad25225fd8 None admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:24.775002 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2cbb62-cbf6-4e61-98d1-b6ad25225fd8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:24.775307 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2cbb62-cbf6-4e61-98d1-b6ad25225fd8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:24.776674 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2cbb62-cbf6-4e61-98d1-b6ad25225fd8 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:24.792295 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd249ad-7fe3-4373-a28c-5f4d3595886e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:24.792917 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd249ad-7fe3-4373-a28c-5f4d3595886e None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:24.793196 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd249ad-7fe3-4373-a28c-5f4d3595886e None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:24.793576 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd249ad-7fe3-4373-a28c-5f4d3595886e None admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:24.794278 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd249ad-7fe3-4373-a28c-5f4d3595886e None admin] RBAC: Adding query filter params (name=service) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:24.795602 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd249ad-7fe3-4373-a28c-5f4d3595886e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:24.796657 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd249ad-7fe3-4373-a28c-5f4d3595886e None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:24.804716 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 330/650] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:00:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:24.805592 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2cbb62-cbf6-4e61-98d1-b6ad25225fd8 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:24.810748 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:24.821120 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 321/651] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Sep 19 16:00:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:24.822002 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd249ad-7fe3-4373-a28c-5f4d3595886e None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:25.000012 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2cbb62-cbf6-4e61-98d1-b6ad25225fd8 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:25.017374 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd249ad-7fe3-4373-a28c-5f4d3595886e None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:25.056550 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ebd5f9-0d5a-400b-b9f6-5e1eb2d2d91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:25.058017 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ebd5f9-0d5a-400b-b9f6-5e1eb2d2d91f None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:25.058465 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ebd5f9-0d5a-400b-b9f6-5e1eb2d2d91f None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:25.058867 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ebd5f9-0d5a-400b-b9f6-5e1eb2d2d91f None admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:25.060538 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ebd5f9-0d5a-400b-b9f6-5e1eb2d2d91f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:25.060945 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ebd5f9-0d5a-400b-b9f6-5e1eb2d2d91f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:25.063098 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ebd5f9-0d5a-400b-b9f6-5e1eb2d2d91f None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:25.076454 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4111a22b-4ddb-42bf-85ca-a066cfdb88e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:25.077323 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4111a22b-4ddb-42bf-85ca-a066cfdb88e9 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:25.077769 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4111a22b-4ddb-42bf-85ca-a066cfdb88e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:25.078212 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4111a22b-4ddb-42bf-85ca-a066cfdb88e9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:25.085523 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 331/652] 10.222.0.57 () {56 vars in 1137 bytes} [Mon Sep 19 16:00:24 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:25.086842 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ebd5f9-0d5a-400b-b9f6-5e1eb2d2d91f None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:25.087410 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4111a22b-4ddb-42bf-85ca-a066cfdb88e9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:25.089092 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4111a22b-4ddb-42bf-85ca-a066cfdb88e9 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:25.089944 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:25.105527 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-4111a22b-4ddb-42bf-85ca-a066cfdb88e9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 19 16:00:25.105527 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:25.105527 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:25.105527 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:25.105527 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:25.105527 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:25.105527 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:25.105527 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:25.105527 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:25.105527 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:25.105527 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:25.105527 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:25.105527 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 16:00:25.105527 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 16:00:25.105527 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 16:00:25.105527 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 16:00:25.107214 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:25.107214 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:25.107214 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 16:00:25.107214 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:00:25.107214 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 16:00:25.107214 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:00:25.107214 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:25.107214 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:25.107214 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:25.107214 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:25.107214 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:25.107214 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:25.107214 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:25.107214 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:25.107214 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:25.107214 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:25.107214 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:25.107214 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:25.108270 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:25.108270 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:25.108270 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 16:00:25.108270 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 16:00:25.108270 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 16:00:25.108270 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 16:00:25.108270 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 16:00:25.108270 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 16:00:25.108270 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 19 16:00:25.108270 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:25.116702 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 322/653] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 16:00:24 2022] GET /identity/v3/users/cinder => generated 84 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:25.117737 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4111a22b-4ddb-42bf-85ca-a066cfdb88e9 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:25.239570 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:25.289113 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ebd5f9-0d5a-400b-b9f6-5e1eb2d2d91f None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:25.311715 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4111a22b-4ddb-42bf-85ca-a066cfdb88e9 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:25.335135 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1ab6b1-62a1-4c7e-9faf-367617a73c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:25.336047 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ab6b1-62a1-4c7e-9faf-367617a73c1b None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:25.336332 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ab6b1-62a1-4c7e-9faf-367617a73c1b None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:25.336605 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ab6b1-62a1-4c7e-9faf-367617a73c1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:25.349065 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1ab6b1-62a1-4c7e-9faf-367617a73c1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:25.350641 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1ab6b1-62a1-4c7e-9faf-367617a73c1b None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:25.357537 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69931aff-5515-40fb-9a4a-4dbd12215973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:25.359421 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69931aff-5515-40fb-9a4a-4dbd12215973 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:25.359979 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69931aff-5515-40fb-9a4a-4dbd12215973 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:25.360461 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69931aff-5515-40fb-9a4a-4dbd12215973 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:25.367727 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-4a1ab6b1-62a1-4c7e-9faf-367617a73c1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:25.367727 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:25.367727 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:25.367727 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:25.367727 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:25.367727 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:25.367727 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:25.367727 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:25.367727 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:25.367727 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:25.367727 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:25.367727 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:25.367727 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:00:25.367727 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:00:25.367727 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:00:25.367727 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:00:25.368739 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:25.368739 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:25.368739 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:25.368739 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:25.368739 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:25.368739 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:25.368739 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:25.368739 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:25.368739 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:25.368739 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:25.368739 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:25.368739 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:25.368739 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:25.368739 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:25.368739 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:25.368739 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:25.368739 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:00:25.369606 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:00:25.369606 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:25.369606 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:25.369606 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 332/654] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:00:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:25.369825 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1ab6b1-62a1-4c7e-9faf-367617a73c1b None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:25.439289 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69931aff-5515-40fb-9a4a-4dbd12215973 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:25.441101 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69931aff-5515-40fb-9a4a-4dbd12215973 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:25.445759 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:25.502077 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:25.508607 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1ab6b1-62a1-4c7e-9faf-367617a73c1b None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:25.548290 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 323/655] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 19 16:00:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 16:00:25.549523 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69931aff-5515-40fb-9a4a-4dbd12215973 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:25.557360 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae34d45-a0bd-401b-ae14-da96838f2dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:25.557976 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae34d45-a0bd-401b-ae14-da96838f2dfe None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:25.558252 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae34d45-a0bd-401b-ae14-da96838f2dfe None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:25.558554 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae34d45-a0bd-401b-ae14-da96838f2dfe None admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:25.559283 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae34d45-a0bd-401b-ae14-da96838f2dfe None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:25.560115 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae34d45-a0bd-401b-ae14-da96838f2dfe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:25.562173 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae34d45-a0bd-401b-ae14-da96838f2dfe None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:25.587699 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 333/656] 10.222.0.57 () {56 vars in 1137 bytes} [Mon Sep 19 16:00:25 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:25.588624 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae34d45-a0bd-401b-ae14-da96838f2dfe None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:25.660945 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69931aff-5515-40fb-9a4a-4dbd12215973 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:25.698923 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1f7770-764b-4891-b9d7-52d9380d8985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:25.699857 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f7770-764b-4891-b9d7-52d9380d8985 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:25.700359 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f7770-764b-4891-b9d7-52d9380d8985 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:25.700813 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f7770-764b-4891-b9d7-52d9380d8985 None admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:25.701628 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1f7770-764b-4891-b9d7-52d9380d8985 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:25.702178 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1f7770-764b-4891-b9d7-52d9380d8985 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:25.703778 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1f7770-764b-4891-b9d7-52d9380d8985 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:25.715795 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 324/657] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:00:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:25.716888 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1f7770-764b-4891-b9d7-52d9380d8985 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:25.720417 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae34d45-a0bd-401b-ae14-da96838f2dfe None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:25.735935 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:25.742312 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:25.768277 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ccd68f-5d0d-4380-9b21-c446fe338838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:25.769195 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ccd68f-5d0d-4380-9b21-c446fe338838 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:25.769622 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ccd68f-5d0d-4380-9b21-c446fe338838 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:25.770037 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ccd68f-5d0d-4380-9b21-c446fe338838 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:25.790711 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ccd68f-5d0d-4380-9b21-c446fe338838 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:25.793270 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ccd68f-5d0d-4380-9b21-c446fe338838 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:25.810912 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-b8ccd68f-5d0d-4380-9b21-c446fe338838 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:25.810912 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:25.810912 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:25.810912 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:25.810912 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:25.810912 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:25.810912 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:25.810912 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:25.810912 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:25.810912 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:25.810912 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:25.810912 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:25.810912 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:00:25.810912 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:00:25.810912 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:00:25.810912 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:00:25.811960 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:25.811960 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:25.811960 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:25.811960 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:25.811960 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:25.811960 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:25.811960 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:25.811960 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:25.811960 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:25.811960 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:25.811960 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:25.811960 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:25.811960 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:25.811960 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:25.811960 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:25.811960 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:25.811960 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:00:25.812891 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:00:25.812891 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:25.812891 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:25.816567 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 334/658] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:00:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:25.817732 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ccd68f-5d0d-4380-9b21-c446fe338838 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:25.855337 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1f7770-764b-4891-b9d7-52d9380d8985 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:25.897528 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cad274f-4b79-4b61-8ab5-cacbb9d59bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:25.898492 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cad274f-4b79-4b61-8ab5-cacbb9d59bf9 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:25.899079 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cad274f-4b79-4b61-8ab5-cacbb9d59bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:25.899504 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cad274f-4b79-4b61-8ab5-cacbb9d59bf9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:25.943002 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ccd68f-5d0d-4380-9b21-c446fe338838 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:25.979196 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122ccc96-c432-462d-929f-e19df02c3a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:25.979856 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ccc96-c432-462d-929f-e19df02c3a5e None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:25.980286 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ccc96-c432-462d-929f-e19df02c3a5e None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:25.980663 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ccc96-c432-462d-929f-e19df02c3a5e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:25.987678 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cad274f-4b79-4b61-8ab5-cacbb9d59bf9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:25.989583 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122ccc96-c432-462d-929f-e19df02c3a5e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:25.989863 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cad274f-4b79-4b61-8ab5-cacbb9d59bf9 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:25.991407 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122ccc96-c432-462d-929f-e19df02c3a5e None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:26.003924 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-122ccc96-c432-462d-929f-e19df02c3a5e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 19 16:00:26.003924 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:26.003924 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:26.003924 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:26.003924 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:26.003924 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:26.003924 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:26.003924 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:26.003924 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:26.003924 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:26.003924 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:26.003924 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:26.003924 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 16:00:26.003924 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 16:00:26.003924 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 16:00:26.003924 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 16:00:26.005147 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:26.005147 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:26.005147 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 16:00:26.005147 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:00:26.005147 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 16:00:26.005147 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:00:26.005147 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:26.005147 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:26.005147 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:26.005147 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:26.005147 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:26.005147 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:26.005147 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:26.005147 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:26.005147 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:26.005147 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:26.005147 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:26.005147 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:26.006301 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:26.006301 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:26.006301 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 16:00:26.006301 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 16:00:26.006301 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 16:00:26.006301 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 16:00:26.006301 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 16:00:26.006301 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 16:00:26.006301 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 19 16:00:26.006301 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:26.012275 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 335/659] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Sep 19 16:00:25 2022] GET /identity/v3/users/neutron => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:26.012275 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122ccc96-c432-462d-929f-e19df02c3a5e None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:26.030310 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:26.050709 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:26.140746 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122ccc96-c432-462d-929f-e19df02c3a5e None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:26.145018 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 325/660] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 19 16:00:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 16:00:26.146178 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cad274f-4b79-4b61-8ab5-cacbb9d59bf9 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:26.178931 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43864268-0578-4415-a7a5-e10ce87471f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:26.179556 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43864268-0578-4415-a7a5-e10ce87471f6 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:26.179839 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43864268-0578-4415-a7a5-e10ce87471f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:26.180117 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43864268-0578-4415-a7a5-e10ce87471f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:26.180778 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43864268-0578-4415-a7a5-e10ce87471f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:26.181085 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43864268-0578-4415-a7a5-e10ce87471f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:26.182518 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43864268-0578-4415-a7a5-e10ce87471f6 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:26.196572 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 336/661] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:00:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:26.197517 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43864268-0578-4415-a7a5-e10ce87471f6 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:26.292648 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cad274f-4b79-4b61-8ab5-cacbb9d59bf9 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:26.296278 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:26.320611 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43864268-0578-4415-a7a5-e10ce87471f6 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:26.322235 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:26.342626 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cccc740-edde-43a3-9432-6dbe8309780e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:26.343357 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cccc740-edde-43a3-9432-6dbe8309780e None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:26.343738 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cccc740-edde-43a3-9432-6dbe8309780e None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:26.344292 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cccc740-edde-43a3-9432-6dbe8309780e None admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:26.344856 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cccc740-edde-43a3-9432-6dbe8309780e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:26.345232 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cccc740-edde-43a3-9432-6dbe8309780e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:26.346804 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cccc740-edde-43a3-9432-6dbe8309780e None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:26.373125 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 326/662] 10.222.0.57 () {56 vars in 1139 bytes} [Mon Sep 19 16:00:26 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:26.373125 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cccc740-edde-43a3-9432-6dbe8309780e None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:26.375485 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9982578c-364b-45de-acd4-3734e7b77728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:26.376242 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9982578c-364b-45de-acd4-3734e7b77728 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:26.376659 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9982578c-364b-45de-acd4-3734e7b77728 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:26.377014 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9982578c-364b-45de-acd4-3734e7b77728 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:26.388267 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9982578c-364b-45de-acd4-3734e7b77728 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:26.389872 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9982578c-364b-45de-acd4-3734e7b77728 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:26.406562 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-9982578c-364b-45de-acd4-3734e7b77728 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:26.406562 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:26.406562 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:26.406562 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:26.406562 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:26.406562 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:26.406562 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:26.406562 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:26.406562 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:26.406562 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:26.406562 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:26.406562 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:26.406562 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:00:26.406562 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:00:26.406562 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:00:26.406562 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:00:26.407697 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:26.407697 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:26.407697 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:26.407697 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:26.407697 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:26.407697 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:26.407697 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:26.407697 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:26.407697 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:26.407697 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:26.407697 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:26.407697 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:26.407697 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:26.407697 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:26.407697 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:26.407697 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:26.407697 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:00:26.408630 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:00:26.408630 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:26.408630 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:26.408630 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 337/663] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:00:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:26.408989 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9982578c-364b-45de-acd4-3734e7b77728 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:26.487459 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cccc740-edde-43a3-9432-6dbe8309780e None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:26.510936 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9982578c-364b-45de-acd4-3734e7b77728 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:26.524387 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e08bdfa-7e9f-4373-b755-0b8559a2e699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:26.525021 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e08bdfa-7e9f-4373-b755-0b8559a2e699 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:26.525303 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e08bdfa-7e9f-4373-b755-0b8559a2e699 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:26.525576 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e08bdfa-7e9f-4373-b755-0b8559a2e699 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:26.533012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e08bdfa-7e9f-4373-b755-0b8559a2e699 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:26.534475 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e08bdfa-7e9f-4373-b755-0b8559a2e699 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:26.546825 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-9e08bdfa-7e9f-4373-b755-0b8559a2e699 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:00:26.546825 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:26.546825 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:26.546825 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:26.546825 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:26.546825 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:26.546825 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:26.546825 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:26.546825 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:26.546825 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:26.546825 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:26.546825 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:26.546825 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 16:00:26.546825 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 16:00:26.546825 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 16:00:26.547756 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 16:00:26.547756 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:26.547756 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:26.547756 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:26.547756 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:26.547756 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:26.547756 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:26.547756 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:26.547756 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:26.547756 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:26.547756 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:26.547756 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:26.547756 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:26.547756 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:26.547756 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:26.547756 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:26.547756 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:26.548668 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 16:00:26.548668 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 16:00:26.548668 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 16:00:26.548668 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 16:00:26.548668 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 16:00:26.548668 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 16:00:26.548668 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:00:26.548668 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:26.551267 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a61472d-5867-4b29-af18-0373a34be7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:26.551876 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a61472d-5867-4b29-af18-0373a34be7ea None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:26.552153 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a61472d-5867-4b29-af18-0373a34be7ea None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:26.552422 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a61472d-5867-4b29-af18-0373a34be7ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:26.552779 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 327/664] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 19 16:00:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:26.553861 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b722946d-c3e0-4687-b348-0355d642745b None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:26.554146 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b722946d-c3e0-4687-b348-0355d642745b None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:26.554444 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b722946d-c3e0-4687-b348-0355d642745b None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:26.555855 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 328/665] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:00:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:00:26.557266 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:26.560836 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b722946d-c3e0-4687-b348-0355d642745b None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:26.609433 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:26.638680 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a61472d-5867-4b29-af18-0373a34be7ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:26.640186 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a61472d-5867-4b29-af18-0373a34be7ea None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:26.707244 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b722946d-c3e0-4687-b348-0355d642745b None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:26.747861 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952eea94-b810-47a7-9203-b37065f1defb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:26.748468 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952eea94-b810-47a7-9203-b37065f1defb None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:26.748750 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952eea94-b810-47a7-9203-b37065f1defb None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:26.749032 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952eea94-b810-47a7-9203-b37065f1defb None admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:26.749162 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 338/666] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 19 16:00:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 16:00:26.749680 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952eea94-b810-47a7-9203-b37065f1defb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:26.749985 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952eea94-b810-47a7-9203-b37065f1defb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:26.750082 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a61472d-5867-4b29-af18-0373a34be7ea None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:26.751460 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952eea94-b810-47a7-9203-b37065f1defb None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:26.763990 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 329/667] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:00:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:26.765310 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa85ef6f-7962-4026-8980-770ccaf63fa8 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:26.765598 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa85ef6f-7962-4026-8980-770ccaf63fa8 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:26.765878 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa85ef6f-7962-4026-8980-770ccaf63fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:26.784453 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-aa85ef6f-7962-4026-8980-770ccaf63fa8 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:00:26.785679 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-aa85ef6f-7962-4026-8980-770ccaf63fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:00:26.792736 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:26.852711 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 330/668] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:00:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 16:00:26.854107 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e05cca-14cb-41d6-bc6b-ebb24bd8ee47 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:26.854325 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e05cca-14cb-41d6-bc6b-ebb24bd8ee47 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:26.854642 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e05cca-14cb-41d6-bc6b-ebb24bd8ee47 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:26.856187 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 331/669] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:00:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:00:26.856929 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e05cca-14cb-41d6-bc6b-ebb24bd8ee47 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:26.857664 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a61472d-5867-4b29-af18-0373a34be7ea None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:26.899302 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37a79ac-d3c0-47a8-8b47-2388f3859710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:26.900137 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:26.901541 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a79ac-d3c0-47a8-8b47-2388f3859710 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:26.901963 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a79ac-d3c0-47a8-8b47-2388f3859710 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:26.902394 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a79ac-d3c0-47a8-8b47-2388f3859710 None admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:26.904866 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37a79ac-d3c0-47a8-8b47-2388f3859710 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:26.905605 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37a79ac-d3c0-47a8-8b47-2388f3859710 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:26.907493 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37a79ac-d3c0-47a8-8b47-2388f3859710 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:26.924461 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 339/670] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 19 16:00:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:26.925656 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37a79ac-d3c0-47a8-8b47-2388f3859710 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:26.986548 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e05cca-14cb-41d6-bc6b-ebb24bd8ee47 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:27.031454 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bb2f06-1200-486e-8b13-a45de433d41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:27.032099 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bb2f06-1200-486e-8b13-a45de433d41c None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:27.032382 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bb2f06-1200-486e-8b13-a45de433d41c None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:27.032657 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bb2f06-1200-486e-8b13-a45de433d41c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:27.044611 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bb2f06-1200-486e-8b13-a45de433d41c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:27.046075 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bb2f06-1200-486e-8b13-a45de433d41c None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:27.059796 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:27.062269 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-08bb2f06-1200-486e-8b13-a45de433d41c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:27.062269 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:27.062269 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:27.062269 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:27.062269 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:27.062269 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:27.062269 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:27.062269 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:27.062269 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:27.062269 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:27.062269 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:27.062269 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:27.062269 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:00:27.062269 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:00:27.062269 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:00:27.062269 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:00:27.063390 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:27.063390 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:27.063390 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:27.063390 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:27.063390 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:27.063390 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:27.063390 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:27.063390 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:27.063390 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:27.063390 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:27.063390 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:27.063390 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:27.063390 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:27.063390 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:27.063390 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:27.063390 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:27.063390 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:00:27.064364 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:00:27.064364 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:27.064364 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:27.068110 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 332/671] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:00:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:27.069448 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d234a0-c659-4778-9963-9bfe646d3d12 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:27.069735 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d234a0-c659-4778-9963-9bfe646d3d12 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:27.070009 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d234a0-c659-4778-9963-9bfe646d3d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:27.070871 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37a79ac-d3c0-47a8-8b47-2388f3859710 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:27.088502 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-24d234a0-c659-4778-9963-9bfe646d3d12 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:00:27.089698 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-24d234a0-c659-4778-9963-9bfe646d3d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:00:27.109659 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c085ba53-2a8e-4477-ad91-a4690f5523a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:27.110560 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085ba53-2a8e-4477-ad91-a4690f5523a8 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:27.111125 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085ba53-2a8e-4477-ad91-a4690f5523a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:27.111398 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085ba53-2a8e-4477-ad91-a4690f5523a8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:27.118572 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c085ba53-2a8e-4477-ad91-a4690f5523a8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:27.120018 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c085ba53-2a8e-4477-ad91-a4690f5523a8 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:27.131794 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-c085ba53-2a8e-4477-ad91-a4690f5523a8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:00:27.131794 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:27.131794 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:27.131794 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:27.131794 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:27.131794 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:27.131794 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:27.131794 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:27.131794 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:27.131794 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:27.131794 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:27.131794 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:27.131794 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 16:00:27.131794 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 16:00:27.131794 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 16:00:27.132982 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 16:00:27.132982 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:27.132982 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:27.132982 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:27.132982 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:27.132982 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:27.132982 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:27.132982 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:27.132982 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:27.132982 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:27.132982 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:27.132982 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:27.132982 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:27.132982 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:27.132982 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:27.132982 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:27.132982 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:27.134335 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 16:00:27.134335 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 16:00:27.134335 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 16:00:27.134335 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 16:00:27.134335 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 16:00:27.134335 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 16:00:27.134335 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:00:27.134335 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:27.134335 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 340/672] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 19 16:00:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:27.134335 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c085ba53-2a8e-4477-ad91-a4690f5523a8 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:27.159672 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 333/673] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:00:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 16:00:27.160508 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d234a0-c659-4778-9963-9bfe646d3d12 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:27.172509 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:27.239428 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c085ba53-2a8e-4477-ad91-a4690f5523a8 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:27.272491 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d234a0-c659-4778-9963-9bfe646d3d12 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:27.274769 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d388b653-6d21-480d-b776-0a5f95bb86ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:27.275452 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d388b653-6d21-480d-b776-0a5f95bb86ee None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:27.275730 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d388b653-6d21-480d-b776-0a5f95bb86ee None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:27.276008 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d388b653-6d21-480d-b776-0a5f95bb86ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:27.276730 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d388b653-6d21-480d-b776-0a5f95bb86ee None admin] RBAC: Adding query filter params (role.id=1e475e6a170a4dbabc4ce8348eeb30b9, scope.project.id=be03e5300b3343169f35c643c93d1e06, user.id=5632ade0cc6f4ae28f72160de68486cb) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:27.277039 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d388b653-6d21-480d-b776-0a5f95bb86ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:27.279083 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d388b653-6d21-480d-b776-0a5f95bb86ee None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:27.290416 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 341/674] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 19 16:00:27 2022] GET /identity/v3/role_assignments?user.id=5632ade0cc6f4ae28f72160de68486cb&scope.project.id=be03e5300b3343169f35c643c93d1e06&role.id=1e475e6a170a4dbabc4ce8348eeb30b9 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:27.290416 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d388b653-6d21-480d-b776-0a5f95bb86ee None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:27.313243 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:27.340245 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109ccf47-6459-4bff-ad39-e5665ac04798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:27.341596 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ccf47-6459-4bff-ad39-e5665ac04798 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:27.341945 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ccf47-6459-4bff-ad39-e5665ac04798 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:27.342283 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ccf47-6459-4bff-ad39-e5665ac04798 None admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:27.343046 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109ccf47-6459-4bff-ad39-e5665ac04798 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:27.343421 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109ccf47-6459-4bff-ad39-e5665ac04798 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:27.344893 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109ccf47-6459-4bff-ad39-e5665ac04798 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:27.358941 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 334/675] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:00:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:27.360092 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109ccf47-6459-4bff-ad39-e5665ac04798 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:27.442193 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:27.445283 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d388b653-6d21-480d-b776-0a5f95bb86ee None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:27.477591 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109ccf47-6459-4bff-ad39-e5665ac04798 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:27.487029 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92183ae8-8b40-4497-a90c-e7563496b1a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:27.487207 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92183ae8-8b40-4497-a90c-e7563496b1a2 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:27.487428 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92183ae8-8b40-4497-a90c-e7563496b1a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:27.487705 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92183ae8-8b40-4497-a90c-e7563496b1a2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:27.503380 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92183ae8-8b40-4497-a90c-e7563496b1a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 16:00:27.508634 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-92183ae8-8b40-4497-a90c-e7563496b1a2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 16:00:27.508634 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:27.508634 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:27.508634 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:27.508634 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:27.508634 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:27.508634 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:27.508634 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:27.508634 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:27.508634 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:27.508634 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:27.508634 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:27.508634 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 16:00:27.508634 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 16:00:27.508634 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 16:00:27.508634 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 16:00:27.509520 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 16:00:27.509520 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 16:00:27.509520 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 16:00:27.509520 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 16:00:27.509520 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:27.509520 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:27.509520 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:27.509520 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:27.509520 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:27.509520 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:27.509520 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:27.509520 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:27.509520 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:27.509520 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:27.509520 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:27.509520 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:27.510365 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:27.510365 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:27.510365 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:27.510365 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:27.510365 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 16:00:27.510365 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 16:00:27.510365 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 16:00:27.510365 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 16:00:27.510365 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 16:00:27.510365 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 16:00:27.510365 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 16:00:27.510365 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:27.510365 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 342/676] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 19 16:00:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:27.511069 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92183ae8-8b40-4497-a90c-e7563496b1a2 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:27.521639 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc46ada-bc21-45c2-8bcc-17e05e3d62a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:27.522262 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc46ada-bc21-45c2-8bcc-17e05e3d62a2 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:27.522565 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc46ada-bc21-45c2-8bcc-17e05e3d62a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:27.522843 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc46ada-bc21-45c2-8bcc-17e05e3d62a2 None admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:27.523533 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc46ada-bc21-45c2-8bcc-17e05e3d62a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:27.523839 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc46ada-bc21-45c2-8bcc-17e05e3d62a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:27.525252 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc46ada-bc21-45c2-8bcc-17e05e3d62a2 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:27.544589 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 335/677] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 19 16:00:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:27.544589 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc46ada-bc21-45c2-8bcc-17e05e3d62a2 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:27.603146 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:27.625019 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92183ae8-8b40-4497-a90c-e7563496b1a2 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:27.652167 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc46ada-bc21-45c2-8bcc-17e05e3d62a2 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:27.659738 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbb9ca5-162c-4a2d-af74-c6ceb4df126c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:27.660374 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbb9ca5-162c-4a2d-af74-c6ceb4df126c None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:27.660649 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbb9ca5-162c-4a2d-af74-c6ceb4df126c None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:27.660918 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbb9ca5-162c-4a2d-af74-c6ceb4df126c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:27.669973 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbb9ca5-162c-4a2d-af74-c6ceb4df126c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:27.672056 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbb9ca5-162c-4a2d-af74-c6ceb4df126c None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:27.683684 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:27.688145 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-1fbb9ca5-162c-4a2d-af74-c6ceb4df126c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:00:27.688145 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:27.688145 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:27.688145 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:27.688145 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:27.688145 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:27.688145 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:27.688145 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:27.688145 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:27.688145 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:27.688145 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:27.688145 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:27.688145 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 16:00:27.688145 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 16:00:27.688145 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 16:00:27.689084 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 16:00:27.689084 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:27.689084 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:27.689084 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:27.689084 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:27.689084 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:27.689084 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:27.689084 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:27.689084 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:27.689084 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:27.689084 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:27.689084 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:27.689084 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:27.689084 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:27.689084 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:27.689084 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:27.689084 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:27.689981 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 16:00:27.689981 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 16:00:27.689981 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 16:00:27.689981 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 16:00:27.689981 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 16:00:27.689981 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 16:00:27.689981 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:00:27.689981 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:27.690441 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3734139-1490-4a0d-87db-a6c7ec449a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:27.691013 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3734139-1490-4a0d-87db-a6c7ec449a3f None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:27.691287 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3734139-1490-4a0d-87db-a6c7ec449a3f None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:27.691869 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3734139-1490-4a0d-87db-a6c7ec449a3f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:27.692104 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 343/678] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 19 16:00:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:27.693022 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbb9ca5-162c-4a2d-af74-c6ceb4df126c None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:27.704264 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3734139-1490-4a0d-87db-a6c7ec449a3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 16:00:27.709627 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-e3734139-1490-4a0d-87db-a6c7ec449a3f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 16:00:27.709627 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:27.709627 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:27.709627 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:27.709627 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:27.709627 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:27.709627 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:27.709627 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:27.709627 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:27.709627 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:27.709627 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:27.709627 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:27.709627 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 16:00:27.709627 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 16:00:27.709627 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 16:00:27.709627 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 16:00:27.710841 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 16:00:27.710841 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 16:00:27.710841 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 16:00:27.710841 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 16:00:27.710841 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:27.710841 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:27.710841 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:27.710841 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:27.710841 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:27.710841 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:27.710841 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:27.710841 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:27.710841 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:27.710841 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:27.710841 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:27.710841 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:27.711870 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:27.711870 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:27.711870 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:27.711870 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:27.711870 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 16:00:27.711870 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 16:00:27.711870 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 16:00:27.711870 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 16:00:27.711870 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 16:00:27.711870 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 16:00:27.711870 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 16:00:27.711870 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:27.711870 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 336/679] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Sep 19 16:00:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:27.712791 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3734139-1490-4a0d-87db-a6c7ec449a3f None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:27.812742 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbb9ca5-162c-4a2d-af74-c6ceb4df126c None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:27.825651 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3734139-1490-4a0d-87db-a6c7ec449a3f None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:27.851021 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e846c0-c57b-45d0-95d9-9b80647b9352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:27.851722 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e846c0-c57b-45d0-95d9-9b80647b9352 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:27.852069 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e846c0-c57b-45d0-95d9-9b80647b9352 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:27.852410 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e846c0-c57b-45d0-95d9-9b80647b9352 None admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:27.853075 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e846c0-c57b-45d0-95d9-9b80647b9352 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:27.853379 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e846c0-c57b-45d0-95d9-9b80647b9352 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:27.854766 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e846c0-c57b-45d0-95d9-9b80647b9352 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:27.864484 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c02d714-2a46-46e9-966b-61f25f496d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:27.865299 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c02d714-2a46-46e9-966b-61f25f496d58 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:27.865427 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 344/680] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 16:00:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:27.865533 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c02d714-2a46-46e9-966b-61f25f496d58 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:27.865745 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c02d714-2a46-46e9-966b-61f25f496d58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:27.866351 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c02d714-2a46-46e9-966b-61f25f496d58 None admin] RBAC: Adding query filter params (role.id=1e475e6a170a4dbabc4ce8348eeb30b9, scope.project.id=be03e5300b3343169f35c643c93d1e06, user.id=7d596e43cc7446d29e2d62d9052d6091) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:27.866699 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c02d714-2a46-46e9-966b-61f25f496d58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:27.868121 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c02d714-2a46-46e9-966b-61f25f496d58 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:27.870487 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e846c0-c57b-45d0-95d9-9b80647b9352 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:27.873961 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:27.875557 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 337/681] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 19 16:00:27 2022] GET /identity/v3/role_assignments?user.id=7d596e43cc7446d29e2d62d9052d6091&scope.project.id=be03e5300b3343169f35c643c93d1e06&role.id=1e475e6a170a4dbabc4ce8348eeb30b9 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:27.876211 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c02d714-2a46-46e9-966b-61f25f496d58 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:27.953175 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:28.010300 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e846c0-c57b-45d0-95d9-9b80647b9352 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:28.024796 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c02d714-2a46-46e9-966b-61f25f496d58 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:28.050933 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2da03b-a093-4adf-8ab8-274cf475acd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:28.051474 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2da03b-a093-4adf-8ab8-274cf475acd5 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:28.051755 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2da03b-a093-4adf-8ab8-274cf475acd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:28.052031 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2da03b-a093-4adf-8ab8-274cf475acd5 None admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:28.052734 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2da03b-a093-4adf-8ab8-274cf475acd5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:28.053271 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2da03b-a093-4adf-8ab8-274cf475acd5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:28.054612 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2da03b-a093-4adf-8ab8-274cf475acd5 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:28.071016 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 345/682] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 19 16:00:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:28.071730 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2da03b-a093-4adf-8ab8-274cf475acd5 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:28.085726 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0beaba-9224-4e07-9c50-23239132a1b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:28.086552 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0beaba-9224-4e07-9c50-23239132a1b4 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:28.086835 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0beaba-9224-4e07-9c50-23239132a1b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:28.087104 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0beaba-9224-4e07-9c50-23239132a1b4 None admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:28.087775 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0beaba-9224-4e07-9c50-23239132a1b4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:28.088082 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0beaba-9224-4e07-9c50-23239132a1b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:28.089562 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0beaba-9224-4e07-9c50-23239132a1b4 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:28.100692 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 338/683] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Sep 19 16:00:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:28.101385 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0beaba-9224-4e07-9c50-23239132a1b4 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:28.153759 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:28.205490 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2da03b-a093-4adf-8ab8-274cf475acd5 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:28.239737 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0beaba-9224-4e07-9c50-23239132a1b4 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:28.245709 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ae79d7-0884-4baa-bf47-2a49116f20f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:28.246666 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ae79d7-0884-4baa-bf47-2a49116f20f8 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:28.247697 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ae79d7-0884-4baa-bf47-2a49116f20f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:28.248211 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ae79d7-0884-4baa-bf47-2a49116f20f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:28.259624 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:28.260387 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ae79d7-0884-4baa-bf47-2a49116f20f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:28.262232 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ae79d7-0884-4baa-bf47-2a49116f20f8 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:28.279922 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-30ae79d7-0884-4baa-bf47-2a49116f20f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:28.279922 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:28.279922 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:28.279922 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:28.279922 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:28.279922 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:28.279922 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:28.279922 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:28.279922 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:28.279922 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:28.279922 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:28.279922 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:28.279922 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:00:28.279922 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:00:28.279922 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:00:28.279922 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:00:28.281102 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:28.281102 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:28.281102 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:28.281102 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:28.281102 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:28.281102 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:28.281102 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:28.281102 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:28.281102 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:28.281102 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:28.281102 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:28.281102 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:28.281102 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:28.281102 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:28.281102 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:28.281102 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:28.281102 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:00:28.282210 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:00:28.282210 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:28.282210 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:28.282445 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20203fe2-d45f-4813-92a1-a8dc78b47cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:28.283031 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20203fe2-d45f-4813-92a1-a8dc78b47cae None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:28.283312 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20203fe2-d45f-4813-92a1-a8dc78b47cae None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:28.283586 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20203fe2-d45f-4813-92a1-a8dc78b47cae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:28.288849 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20203fe2-d45f-4813-92a1-a8dc78b47cae None admin] RBAC: Adding query filter params (role.id=1e475e6a170a4dbabc4ce8348eeb30b9, scope.project.id=be03e5300b3343169f35c643c93d1e06, user.id=50355d7b1c6c4f0bbbbc7fe56efb9039) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:28.289049 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 346/684] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:00:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:28.289132 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20203fe2-d45f-4813-92a1-a8dc78b47cae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:28.289230 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ae79d7-0884-4baa-bf47-2a49116f20f8 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:28.289339 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20203fe2-d45f-4813-92a1-a8dc78b47cae None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:28.295176 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 339/685] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 19 16:00:28 2022] GET /identity/v3/role_assignments?user.id=50355d7b1c6c4f0bbbbc7fe56efb9039&scope.project.id=be03e5300b3343169f35c643c93d1e06&role.id=1e475e6a170a4dbabc4ce8348eeb30b9 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:28.296198 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20203fe2-d45f-4813-92a1-a8dc78b47cae None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:28.515424 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ae79d7-0884-4baa-bf47-2a49116f20f8 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:28.516891 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20203fe2-d45f-4813-92a1-a8dc78b47cae None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:28.535083 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:28.563169 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243fb1e8-076b-4027-8136-c598d88f84bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:28.563848 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243fb1e8-076b-4027-8136-c598d88f84bf None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:28.564081 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243fb1e8-076b-4027-8136-c598d88f84bf None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:28.564353 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243fb1e8-076b-4027-8136-c598d88f84bf None admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:28.565028 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243fb1e8-076b-4027-8136-c598d88f84bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:28.565333 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243fb1e8-076b-4027-8136-c598d88f84bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:28.566743 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243fb1e8-076b-4027-8136-c598d88f84bf None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:28.568632 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f35ad6-3c13-4120-b09e-82d22e3716fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:28.569682 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f35ad6-3c13-4120-b09e-82d22e3716fa None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:28.570193 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f35ad6-3c13-4120-b09e-82d22e3716fa None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:28.570685 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f35ad6-3c13-4120-b09e-82d22e3716fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:28.581131 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 340/686] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:00:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:28.586897 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f35ad6-3c13-4120-b09e-82d22e3716fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:28.587220 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243fb1e8-076b-4027-8136-c598d88f84bf None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:28.588836 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f35ad6-3c13-4120-b09e-82d22e3716fa None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:28.606947 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-56f35ad6-3c13-4120-b09e-82d22e3716fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:28.606947 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:28.606947 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:28.606947 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:28.606947 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:28.606947 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:28.606947 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:28.606947 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:28.606947 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:28.606947 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:28.606947 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:28.606947 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:28.606947 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:00:28.606947 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:00:28.606947 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:00:28.606947 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:00:28.608160 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:28.608160 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:28.608160 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:28.608160 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:28.608160 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:28.608160 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:28.608160 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:28.608160 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:28.608160 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:28.608160 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:28.608160 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:28.608160 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:28.608160 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:28.608160 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:28.608160 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:28.608160 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:28.608160 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:00:28.609257 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:00:28.609257 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:28.609257 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:28.610926 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 347/687] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:00:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:28.617693 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f35ad6-3c13-4120-b09e-82d22e3716fa None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:28.650989 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:28.772949 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243fb1e8-076b-4027-8136-c598d88f84bf None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:28.808423 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f35ad6-3c13-4120-b09e-82d22e3716fa None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:28.826279 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d24a0-5c17-4140-a3ba-dccdc0b6a9e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:28.831907 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d24a0-5c17-4140-a3ba-dccdc0b6a9e9 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:28.832185 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d24a0-5c17-4140-a3ba-dccdc0b6a9e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:28.832454 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d24a0-5c17-4140-a3ba-dccdc0b6a9e9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:28.849503 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04d24a0-5c17-4140-a3ba-dccdc0b6a9e9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:28.850982 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04d24a0-5c17-4140-a3ba-dccdc0b6a9e9 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:28.870193 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-f04d24a0-5c17-4140-a3ba-dccdc0b6a9e9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 19 16:00:28.870193 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:28.870193 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:28.870193 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:28.870193 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:28.870193 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:28.870193 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:28.870193 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:28.870193 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:28.870193 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:28.870193 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:28.870193 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:28.870193 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 16:00:28.870193 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 16:00:28.870193 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 16:00:28.870193 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 16:00:28.871409 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:28.871409 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:28.871409 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 16:00:28.871409 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:00:28.871409 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 16:00:28.871409 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:00:28.871409 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:28.871409 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:28.871409 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:28.871409 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:28.871409 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:28.871409 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:28.871409 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:28.871409 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:28.871409 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:28.871409 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:28.871409 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:28.871409 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:28.872456 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:28.872456 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:28.872456 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 16:00:28.872456 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 16:00:28.872456 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 16:00:28.872456 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 16:00:28.872456 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 16:00:28.872456 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 16:00:28.872456 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 19 16:00:28.872456 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:28.874031 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 341/688] 10.222.0.57 () {56 vars in 1093 bytes} [Mon Sep 19 16:00:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 16:00:28.877617 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c868f8d3-4b67-494e-aea2-e3b4118c0f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:28.877617 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c868f8d3-4b67-494e-aea2-e3b4118c0f7c None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:28.877617 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c868f8d3-4b67-494e-aea2-e3b4118c0f7c None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:28.877617 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c868f8d3-4b67-494e-aea2-e3b4118c0f7c None admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:28.878327 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c868f8d3-4b67-494e-aea2-e3b4118c0f7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:28.878327 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c868f8d3-4b67-494e-aea2-e3b4118c0f7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:28.878327 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c868f8d3-4b67-494e-aea2-e3b4118c0f7c None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:28.880331 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d24a0-5c17-4140-a3ba-dccdc0b6a9e9 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:28.901157 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 348/689] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:00:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:28.907432 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c868f8d3-4b67-494e-aea2-e3b4118c0f7c None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:28.948630 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:29.070879 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:29.074166 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d24a0-5c17-4140-a3ba-dccdc0b6a9e9 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:29.125857 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c868f8d3-4b67-494e-aea2-e3b4118c0f7c None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:29.135329 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf181c2-cd05-4322-b021-76a22785d9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:29.135954 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf181c2-cd05-4322-b021-76a22785d9fc None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:29.136230 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf181c2-cd05-4322-b021-76a22785d9fc None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:29.136504 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf181c2-cd05-4322-b021-76a22785d9fc None admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:29.137249 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf181c2-cd05-4322-b021-76a22785d9fc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:29.137589 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf181c2-cd05-4322-b021-76a22785d9fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:29.139055 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf181c2-cd05-4322-b021-76a22785d9fc None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:29.160731 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 342/690] 10.222.0.57 () {56 vars in 1133 bytes} [Mon Sep 19 16:00:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:29.166104 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf181c2-cd05-4322-b021-76a22785d9fc None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:29.195099 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114ce1c1-dd0a-4ba6-aefd-387730e46859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:29.195743 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ce1c1-dd0a-4ba6-aefd-387730e46859 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:29.196018 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ce1c1-dd0a-4ba6-aefd-387730e46859 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:29.196287 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ce1c1-dd0a-4ba6-aefd-387730e46859 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:29.209715 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114ce1c1-dd0a-4ba6-aefd-387730e46859 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:29.211195 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114ce1c1-dd0a-4ba6-aefd-387730e46859 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:29.232284 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-114ce1c1-dd0a-4ba6-aefd-387730e46859 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 19 16:00:29.232284 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:29.232284 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:29.232284 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:29.232284 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:29.232284 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:29.232284 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:29.232284 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:29.232284 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:29.232284 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:29.232284 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:29.232284 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:29.232284 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 16:00:29.232284 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 16:00:29.232284 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 16:00:29.232284 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 16:00:29.233273 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:29.233273 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:29.233273 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 16:00:29.233273 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:00:29.233273 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 16:00:29.233273 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:00:29.233273 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:29.233273 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:29.233273 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:29.233273 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:29.233273 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:29.233273 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:29.233273 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:29.233273 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:29.233273 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:29.233273 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:29.233273 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:29.233273 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:29.234255 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:29.234255 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:29.234255 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 16:00:29.234255 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 16:00:29.234255 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 16:00:29.234255 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 16:00:29.234255 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 16:00:29.234255 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 16:00:29.234255 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 19 16:00:29.234255 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:29.243642 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 349/691] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 19 16:00:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:29.251440 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114ce1c1-dd0a-4ba6-aefd-387730e46859 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:29.311168 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf181c2-cd05-4322-b021-76a22785d9fc None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:29.350758 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0de33b-cf33-4a6a-a154-8938f16627fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:29.351300 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0de33b-cf33-4a6a-a154-8938f16627fe None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:29.351575 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0de33b-cf33-4a6a-a154-8938f16627fe None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:29.351847 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0de33b-cf33-4a6a-a154-8938f16627fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:29.426782 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:29.436524 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:29.471262 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114ce1c1-dd0a-4ba6-aefd-387730e46859 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:29.503448 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0de33b-cf33-4a6a-a154-8938f16627fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:29.504880 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0de33b-cf33-4a6a-a154-8938f16627fe None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:29.543213 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12863c6-3c30-4838-8447-27bf88317a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:29.543860 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12863c6-3c30-4838-8447-27bf88317a12 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:29.544136 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12863c6-3c30-4838-8447-27bf88317a12 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:29.544404 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12863c6-3c30-4838-8447-27bf88317a12 None admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:29.545140 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12863c6-3c30-4838-8447-27bf88317a12 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:29.545443 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12863c6-3c30-4838-8447-27bf88317a12 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:29.546892 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12863c6-3c30-4838-8447-27bf88317a12 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:29.582879 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 350/692] 10.222.0.57 () {56 vars in 1135 bytes} [Mon Sep 19 16:00:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:29.591612 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12863c6-3c30-4838-8447-27bf88317a12 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:29.657764 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 343/693] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 19 16:00:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 16:00:29.663503 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0de33b-cf33-4a6a-a154-8938f16627fe None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:29.766799 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:29.789334 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0de33b-cf33-4a6a-a154-8938f16627fe None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:29.821023 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12863c6-3c30-4838-8447-27bf88317a12 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:29.838806 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de56e32-eb70-447f-ba7a-8a07f424a1f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:29.839553 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de56e32-eb70-447f-ba7a-8a07f424a1f5 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:29.839823 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de56e32-eb70-447f-ba7a-8a07f424a1f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:29.840094 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de56e32-eb70-447f-ba7a-8a07f424a1f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:29.851052 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de56e32-eb70-447f-ba7a-8a07f424a1f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:29.852658 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de56e32-eb70-447f-ba7a-8a07f424a1f5 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:29.864906 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b469efdb-5ea6-4263-b96a-3069dadd3fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:29.865615 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b469efdb-5ea6-4263-b96a-3069dadd3fed None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:29.865965 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b469efdb-5ea6-4263-b96a-3069dadd3fed None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:29.866313 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b469efdb-5ea6-4263-b96a-3069dadd3fed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:29.868821 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-7de56e32-eb70-447f-ba7a-8a07f424a1f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:29.868821 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:29.868821 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:29.868821 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:29.868821 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:29.868821 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:29.868821 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:29.868821 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:29.868821 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:29.868821 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:29.868821 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:29.868821 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:29.868821 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:00:29.868821 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:00:29.868821 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:00:29.868821 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:00:29.869748 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:29.869748 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:29.869748 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:29.869748 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:29.869748 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:29.869748 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:29.869748 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:29.869748 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:29.869748 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:29.869748 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:29.869748 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:29.869748 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:29.869748 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:29.869748 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:29.869748 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:29.869748 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:29.869748 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:00:29.870694 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:00:29.870694 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:29.870694 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:29.874742 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 344/694] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:00:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:29.876025 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98405a8d-b13f-4032-ad06-a635207018d4 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:29.876309 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98405a8d-b13f-4032-ad06-a635207018d4 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:29.876579 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98405a8d-b13f-4032-ad06-a635207018d4 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:29.878152 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 345/695] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:00:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:00:29.882880 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:29.884437 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98405a8d-b13f-4032-ad06-a635207018d4 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:29.955295 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b469efdb-5ea6-4263-b96a-3069dadd3fed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:29.956746 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b469efdb-5ea6-4263-b96a-3069dadd3fed None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:30.011664 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98405a8d-b13f-4032-ad06-a635207018d4 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:30.026883 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:30.062316 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3deb64-d129-4e4c-897b-c944b2157263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:30.063283 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3deb64-d129-4e4c-897b-c944b2157263 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:30.063660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3deb64-d129-4e4c-897b-c944b2157263 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:30.064042 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3deb64-d129-4e4c-897b-c944b2157263 None admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:30.064964 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3deb64-d129-4e4c-897b-c944b2157263 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:30.065275 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3deb64-d129-4e4c-897b-c944b2157263 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:30.066665 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3deb64-d129-4e4c-897b-c944b2157263 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:30.081149 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 351/696] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 19 16:00:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 19 16:00:30.082823 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19acfc5-c13c-4d3b-a650-f85968d53626 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:30.083332 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19acfc5-c13c-4d3b-a650-f85968d53626 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:30.083828 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19acfc5-c13c-4d3b-a650-f85968d53626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:30.091042 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 346/697] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:00:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:30.091042 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3deb64-d129-4e4c-897b-c944b2157263 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:30.111279 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-a19acfc5-c13c-4d3b-a650-f85968d53626 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:00:30.112645 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-a19acfc5-c13c-4d3b-a650-f85968d53626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:00:30.162041 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:30.198680 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 352/698] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:00:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 16:00:30.200514 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19acfc5-c13c-4d3b-a650-f85968d53626 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:30.216615 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3deb64-d129-4e4c-897b-c944b2157263 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:30.255201 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3fe5d5-23f4-42a9-b913-ac8477ef6d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:30.256082 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3fe5d5-23f4-42a9-b913-ac8477ef6d3b None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:30.256369 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3fe5d5-23f4-42a9-b913-ac8477ef6d3b None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:30.256644 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3fe5d5-23f4-42a9-b913-ac8477ef6d3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:30.269731 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3fe5d5-23f4-42a9-b913-ac8477ef6d3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:30.271340 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3fe5d5-23f4-42a9-b913-ac8477ef6d3b None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:30.290881 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-7b3fe5d5-23f4-42a9-b913-ac8477ef6d3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:30.290881 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:30.290881 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:30.290881 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:30.290881 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:30.290881 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:30.290881 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:30.290881 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:30.290881 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:30.290881 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:30.290881 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:30.290881 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:30.290881 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:00:30.290881 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:00:30.290881 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:00:30.290881 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:00:30.292096 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:30.292096 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:30.292096 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:30.292096 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:30.292096 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:30.292096 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:30.292096 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:30.292096 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:30.292096 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:30.292096 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:30.292096 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:30.292096 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:30.292096 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:30.292096 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:30.292096 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:30.292096 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:30.292096 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:00:30.292969 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:00:30.292969 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:30.292969 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:30.292969 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 347/699] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:00:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:30.299387 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3fe5d5-23f4-42a9-b913-ac8477ef6d3b None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:30.330756 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:30.419750 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3fe5d5-23f4-42a9-b913-ac8477ef6d3b None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:30.466949 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a47552-9745-4457-99b5-dc0b7842433a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:30.467482 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a47552-9745-4457-99b5-dc0b7842433a None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:30.467761 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a47552-9745-4457-99b5-dc0b7842433a None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:30.468035 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a47552-9745-4457-99b5-dc0b7842433a None admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:30.468705 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a47552-9745-4457-99b5-dc0b7842433a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:30.469237 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a47552-9745-4457-99b5-dc0b7842433a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:30.471389 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a47552-9745-4457-99b5-dc0b7842433a None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:30.473102 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19acfc5-c13c-4d3b-a650-f85968d53626 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:30.485509 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 348/700] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:00:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:00:30.491111 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1d6919-1b78-4228-98b7-c0398e3b6a4d None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:30.492400 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1d6919-1b78-4228-98b7-c0398e3b6a4d None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:30.492745 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1d6919-1b78-4228-98b7-c0398e3b6a4d None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:30.495311 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 349/701] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:00:30 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:00:30.500393 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1d6919-1b78-4228-98b7-c0398e3b6a4d None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:30.521909 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e780f21e-c58b-4def-b758-c23f41e1eddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:30.522693 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e780f21e-c58b-4def-b758-c23f41e1eddc None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:30.523036 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e780f21e-c58b-4def-b758-c23f41e1eddc None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:30.523382 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e780f21e-c58b-4def-b758-c23f41e1eddc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:30.531109 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e780f21e-c58b-4def-b758-c23f41e1eddc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:30.532550 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e780f21e-c58b-4def-b758-c23f41e1eddc None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:30.544367 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-e780f21e-c58b-4def-b758-c23f41e1eddc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:00:30.544367 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:30.544367 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:30.544367 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:30.544367 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:30.544367 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:30.544367 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:30.544367 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:30.544367 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:30.544367 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:30.544367 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:30.544367 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:30.544367 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 16:00:30.544367 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 16:00:30.544367 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 16:00:30.545311 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 16:00:30.545311 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:30.545311 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:30.545311 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:30.545311 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:30.545311 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:30.545311 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:30.545311 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:30.545311 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:30.545311 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:30.545311 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:30.545311 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:30.545311 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:30.545311 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:30.545311 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:30.545311 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:30.545311 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:30.547886 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 16:00:30.547886 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 16:00:30.547886 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 16:00:30.547886 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 16:00:30.547886 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 16:00:30.547886 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 16:00:30.547886 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:00:30.547886 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:30.547886 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 353/702] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 19 16:00:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:30.547886 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92081c13-cb9e-41d2-8d86-8ad9713e1523 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:30.547886 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92081c13-cb9e-41d2-8d86-8ad9713e1523 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:30.548596 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92081c13-cb9e-41d2-8d86-8ad9713e1523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:30.566814 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-92081c13-cb9e-41d2-8d86-8ad9713e1523 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:00:30.568084 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-92081c13-cb9e-41d2-8d86-8ad9713e1523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:00:30.595455 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:30.597238 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:30.623490 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1d6919-1b78-4228-98b7-c0398e3b6a4d None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:30.644228 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 354/703] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:00:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 16:00:30.651650 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92081c13-cb9e-41d2-8d86-8ad9713e1523 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:30.664997 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4524695-e14c-469d-ae6f-e6b9518bbb7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:30.665763 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4524695-e14c-469d-ae6f-e6b9518bbb7f None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:30.666114 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4524695-e14c-469d-ae6f-e6b9518bbb7f None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:30.666886 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4524695-e14c-469d-ae6f-e6b9518bbb7f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:30.675229 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4524695-e14c-469d-ae6f-e6b9518bbb7f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:30.676876 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4524695-e14c-469d-ae6f-e6b9518bbb7f None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:30.690948 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-f4524695-e14c-469d-ae6f-e6b9518bbb7f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:00:30.690948 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:30.690948 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:30.690948 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:30.690948 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:30.690948 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:30.690948 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:30.690948 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:30.690948 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:30.690948 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:30.690948 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:30.690948 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:30.690948 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 16:00:30.690948 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 16:00:30.690948 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 16:00:30.691896 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 16:00:30.691896 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:30.691896 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:30.691896 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:30.691896 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:30.691896 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:30.691896 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:30.691896 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:30.691896 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:30.691896 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:30.691896 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:30.691896 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:30.691896 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:30.691896 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:30.691896 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:30.691896 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:30.691896 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:30.692769 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 16:00:30.692769 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 16:00:30.692769 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 16:00:30.692769 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 16:00:30.692769 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 16:00:30.692769 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 16:00:30.692769 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:00:30.692769 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:30.697139 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 350/704] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 19 16:00:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:30.698117 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4524695-e14c-469d-ae6f-e6b9518bbb7f None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:30.810811 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92081c13-cb9e-41d2-8d86-8ad9713e1523 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:30.834114 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4524695-e14c-469d-ae6f-e6b9518bbb7f None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:30.849249 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694a9761-95f7-4624-932f-d1e91c149f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:30.849883 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a9761-95f7-4624-932f-d1e91c149f86 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:30.850160 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a9761-95f7-4624-932f-d1e91c149f86 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:30.850682 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a9761-95f7-4624-932f-d1e91c149f86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:30.860614 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694a9761-95f7-4624-932f-d1e91c149f86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:30.862060 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694a9761-95f7-4624-932f-d1e91c149f86 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:30.878467 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-694a9761-95f7-4624-932f-d1e91c149f86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:30.878467 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:30.878467 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:30.878467 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:30.878467 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:30.878467 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:30.878467 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:30.878467 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:30.878467 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:30.878467 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:30.878467 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:30.878467 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:30.878467 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:00:30.878467 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:00:30.878467 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:00:30.878467 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:00:30.879754 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:30.879754 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:30.879754 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:30.879754 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:30.879754 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:30.879754 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:30.879754 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:30.879754 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:30.879754 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:30.879754 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:30.879754 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:30.879754 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:30.879754 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:30.879754 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:30.879754 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:30.879754 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:30.879754 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:00:30.881153 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:00:30.881153 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:30.881153 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:30.881153 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 355/705] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:00:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:30.881153 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694a9761-95f7-4624-932f-d1e91c149f86 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:30.897243 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7af15d-cb18-4c55-b526-33c19951c1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:30.898198 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7af15d-cb18-4c55-b526-33c19951c1e2 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:30.898198 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7af15d-cb18-4c55-b526-33c19951c1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:30.898417 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7af15d-cb18-4c55-b526-33c19951c1e2 None admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:30.899125 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7af15d-cb18-4c55-b526-33c19951c1e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:30.899429 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7af15d-cb18-4c55-b526-33c19951c1e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:30.900900 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7af15d-cb18-4c55-b526-33c19951c1e2 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:30.901773 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:30.920670 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 351/706] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 19 16:00:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:30.921978 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0072fdc-6bc8-4c84-8601-33e18c380e07 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:30.922256 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0072fdc-6bc8-4c84-8601-33e18c380e07 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:30.922611 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0072fdc-6bc8-4c84-8601-33e18c380e07 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:30.923981 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 352/707] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:00:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:00:30.924757 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0072fdc-6bc8-4c84-8601-33e18c380e07 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:30.927703 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:30.996778 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694a9761-95f7-4624-932f-d1e91c149f86 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:31.033093 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0072fdc-6bc8-4c84-8601-33e18c380e07 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:31.033526 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5995c8db-31f6-4c9b-b1de-5f191bbdbc37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:31.034146 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5995c8db-31f6-4c9b-b1de-5f191bbdbc37 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:31.034521 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5995c8db-31f6-4c9b-b1de-5f191bbdbc37 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:31.034825 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5995c8db-31f6-4c9b-b1de-5f191bbdbc37 None admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:31.036386 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5995c8db-31f6-4c9b-b1de-5f191bbdbc37 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:31.036386 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5995c8db-31f6-4c9b-b1de-5f191bbdbc37 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:31.037263 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5995c8db-31f6-4c9b-b1de-5f191bbdbc37 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:31.050803 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 356/708] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 19 16:00:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:31.051697 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5995c8db-31f6-4c9b-b1de-5f191bbdbc37 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:31.074652 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43feb297-fc32-4e78-b56e-4c77b5dfe992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:31.075343 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43feb297-fc32-4e78-b56e-4c77b5dfe992 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:31.075621 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43feb297-fc32-4e78-b56e-4c77b5dfe992 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:31.075893 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43feb297-fc32-4e78-b56e-4c77b5dfe992 None admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:31.076535 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43feb297-fc32-4e78-b56e-4c77b5dfe992 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:31.076843 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43feb297-fc32-4e78-b56e-4c77b5dfe992 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:31.078189 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43feb297-fc32-4e78-b56e-4c77b5dfe992 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:31.089630 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 353/709] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:00:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:31.090482 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43feb297-fc32-4e78-b56e-4c77b5dfe992 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:31.131752 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:31.154352 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:31.154556 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5995c8db-31f6-4c9b-b1de-5f191bbdbc37 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:31.189135 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edef7c04-3f5b-4c4c-a08b-3fb213467304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:31.189758 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edef7c04-3f5b-4c4c-a08b-3fb213467304 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:31.190036 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edef7c04-3f5b-4c4c-a08b-3fb213467304 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:31.190305 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edef7c04-3f5b-4c4c-a08b-3fb213467304 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:31.190916 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edef7c04-3f5b-4c4c-a08b-3fb213467304 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:31.192446 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edef7c04-3f5b-4c4c-a08b-3fb213467304 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:31.195863 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43feb297-fc32-4e78-b56e-4c77b5dfe992 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:31.204068 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-edef7c04-3f5b-4c4c-a08b-3fb213467304 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 19 16:00:31.204068 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:31.204068 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 16:00:31.204068 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 16:00:31.204068 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 16:00:31.204068 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 16:00:31.204068 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 16:00:31.204068 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 16:00:31.204068 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 19 16:00:31.204068 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:31.204068 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 16:00:31.204068 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:31.204068 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:31.204068 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:31.204068 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:31.204068 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:31.204068 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:31.205178 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:31.205178 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:31.205178 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:31.205178 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:31.205178 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:31.205178 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:31.205178 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 16:00:31.205178 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 16:00:31.205178 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 16:00:31.205178 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 16:00:31.205178 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:31.205178 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:31.205178 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:31.205178 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:31.205178 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:31.205178 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:31.206033 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:31.206033 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:31.206033 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:31.206033 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:31.206033 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:31.206033 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:31.206033 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:31.206033 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:31.206033 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:31.206033 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:31.206033 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 16:00:31.206033 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 16:00:31.206033 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 19 16:00:31.206033 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:31.206033 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 357/710] 10.222.0.57 () {56 vars in 1129 bytes} [Mon Sep 19 16:00:31 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:31.206899 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18f1678-630a-4824-a71c-ec15d7b7d659 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:31.206899 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18f1678-630a-4824-a71c-ec15d7b7d659 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:31.206899 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18f1678-630a-4824-a71c-ec15d7b7d659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:31.224618 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-f18f1678-630a-4824-a71c-ec15d7b7d659 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:00:31.225823 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-f18f1678-630a-4824-a71c-ec15d7b7d659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:00:31.233753 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc39cf79-2bb6-4f8f-af9f-834d8b3cb720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:31.234358 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39cf79-2bb6-4f8f-af9f-834d8b3cb720 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:31.234662 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39cf79-2bb6-4f8f-af9f-834d8b3cb720 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:31.234934 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39cf79-2bb6-4f8f-af9f-834d8b3cb720 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:31.235651 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc39cf79-2bb6-4f8f-af9f-834d8b3cb720 None admin] RBAC: Adding query filter params (role.id=63d2d197bfd6497583e94bfb065ce29b, scope.project.id=be03e5300b3343169f35c643c93d1e06, user.id=1ecd01cc72834b72b5ff2c4b47249737) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:31.235959 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc39cf79-2bb6-4f8f-af9f-834d8b3cb720 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:31.237366 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc39cf79-2bb6-4f8f-af9f-834d8b3cb720 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:31.242752 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 354/711] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 19 16:00:31 2022] GET /identity/v3/role_assignments?user.id=1ecd01cc72834b72b5ff2c4b47249737&scope.project.id=be03e5300b3343169f35c643c93d1e06&role.id=63d2d197bfd6497583e94bfb065ce29b => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:31.243536 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc39cf79-2bb6-4f8f-af9f-834d8b3cb720 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:31.318493 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 358/712] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:00:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 16:00:31.318493 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18f1678-630a-4824-a71c-ec15d7b7d659 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:31.383369 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc39cf79-2bb6-4f8f-af9f-834d8b3cb720 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:31.407759 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:31.420433 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:31.434350 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba1866-5481-4dd5-a227-f70379a24149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:31.435001 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba1866-5481-4dd5-a227-f70379a24149 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:31.435282 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba1866-5481-4dd5-a227-f70379a24149 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:31.435554 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba1866-5481-4dd5-a227-f70379a24149 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:31.436278 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ba1866-5481-4dd5-a227-f70379a24149 None admin] RBAC: Adding query filter params (role.id=1e475e6a170a4dbabc4ce8348eeb30b9, scope.project.id=be03e5300b3343169f35c643c93d1e06, user.id=884a0524aca744b99c7496c11c3291ae) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:31.436589 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ba1866-5481-4dd5-a227-f70379a24149 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:31.438008 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ba1866-5481-4dd5-a227-f70379a24149 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:31.446528 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 355/713] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 19 16:00:31 2022] GET /identity/v3/role_assignments?user.id=884a0524aca744b99c7496c11c3291ae&scope.project.id=be03e5300b3343169f35c643c93d1e06&role.id=1e475e6a170a4dbabc4ce8348eeb30b9 => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:31.447614 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba1866-5481-4dd5-a227-f70379a24149 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:31.457836 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18f1678-630a-4824-a71c-ec15d7b7d659 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:31.532434 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fe4d5f-7c39-4b92-ba19-6124405d24a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:31.533486 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fe4d5f-7c39-4b92-ba19-6124405d24a3 None admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:31.533961 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fe4d5f-7c39-4b92-ba19-6124405d24a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:31.534457 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fe4d5f-7c39-4b92-ba19-6124405d24a3 None admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:31.535606 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fe4d5f-7c39-4b92-ba19-6124405d24a3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:31.543254 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fe4d5f-7c39-4b92-ba19-6124405d24a3 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:31.594467 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 359/714] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Sep 19 16:00:31 2022] POST /identity/v3/users => generated 254 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:00:31.608652 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fe4d5f-7c39-4b92-ba19-6124405d24a3 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:31.631691 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba1866-5481-4dd5-a227-f70379a24149 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:31.701705 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e963bb51-d48b-4b24-a7db-3879040a9eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:31.702330 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963bb51-d48b-4b24-a7db-3879040a9eab None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:31.702648 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963bb51-d48b-4b24-a7db-3879040a9eab None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:31.702924 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963bb51-d48b-4b24-a7db-3879040a9eab None admin] PATH_INFO: `/v3/services` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:31.703574 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e963bb51-d48b-4b24-a7db-3879040a9eab None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:31.703879 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e963bb51-d48b-4b24-a7db-3879040a9eab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:31.705279 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e963bb51-d48b-4b24-a7db-3879040a9eab None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:31.712445 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 356/715] 10.222.0.57 () {56 vars in 1124 bytes} [Mon Sep 19 16:00:31 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:31.725613 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e963bb51-d48b-4b24-a7db-3879040a9eab None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:31.763151 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fe4d5f-7c39-4b92-ba19-6124405d24a3 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:31.807029 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:31.812367 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:31.827988 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0e2ea7-0dcd-4413-9a52-3e1238142e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:31.828246 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e2ea7-0dcd-4413-9a52-3e1238142e9d None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:31.828534 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e2ea7-0dcd-4413-9a52-3e1238142e9d None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:31.828806 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e2ea7-0dcd-4413-9a52-3e1238142e9d None admin] PATH_INFO: `/v3/services/network` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:31.829398 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0e2ea7-0dcd-4413-9a52-3e1238142e9d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:31.830812 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0e2ea7-0dcd-4413-9a52-3e1238142e9d None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:31.842637 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-8a0e2ea7-0dcd-4413-9a52-3e1238142e9d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 19 16:00:31.842637 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:31.842637 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 16:00:31.842637 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 16:00:31.842637 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 16:00:31.842637 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 16:00:31.842637 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 16:00:31.842637 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 16:00:31.842637 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 19 16:00:31.842637 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:31.842637 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 16:00:31.842637 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:31.842637 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:31.842637 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:31.842637 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:31.842637 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:31.842637 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:31.843890 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:31.843890 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:31.843890 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:31.843890 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:31.843890 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:31.843890 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:31.843890 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 16:00:31.843890 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 16:00:31.843890 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 16:00:31.843890 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 16:00:31.843890 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:31.843890 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:31.843890 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:31.843890 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:31.843890 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:31.843890 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:31.844769 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:31.844769 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:31.844769 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:31.844769 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:31.844769 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:31.844769 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:31.844769 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:31.844769 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:31.844769 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:31.844769 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:31.844769 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 16:00:31.844769 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 16:00:31.844769 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 19 16:00:31.844769 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:31.844769 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 360/716] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 19 16:00:31 2022] GET /identity/v3/services/network => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:31.851213 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0e2ea7-0dcd-4413-9a52-3e1238142e9d None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:31.875932 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e963bb51-d48b-4b24-a7db-3879040a9eab None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:31.916945 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d98dbe-8ece-469d-84c7-25f830745860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:31.917958 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d98dbe-8ece-469d-84c7-25f830745860 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:31.918520 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d98dbe-8ece-469d-84c7-25f830745860 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:31.919023 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d98dbe-8ece-469d-84c7-25f830745860 None admin] PATH_INFO: `/v3/services` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:31.919859 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d98dbe-8ece-469d-84c7-25f830745860 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:31.920346 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d98dbe-8ece-469d-84c7-25f830745860 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:31.922007 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d98dbe-8ece-469d-84c7-25f830745860 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:31.930117 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 357/717] 10.222.0.57 () {56 vars in 1124 bytes} [Mon Sep 19 16:00:31 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:00:31.984815 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0e2ea7-0dcd-4413-9a52-3e1238142e9d None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:32.036842 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945f0a8a-0f54-4540-932f-f382def7164d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:32.037791 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945f0a8a-0f54-4540-932f-f382def7164d None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:32.038278 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945f0a8a-0f54-4540-932f-f382def7164d None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:32.038773 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945f0a8a-0f54-4540-932f-f382def7164d None admin] PATH_INFO: `/v3/services` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:32.039869 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945f0a8a-0f54-4540-932f-f382def7164d None admin] RBAC: Adding query filter params (name=network) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:32.040387 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945f0a8a-0f54-4540-932f-f382def7164d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:32.042336 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945f0a8a-0f54-4540-932f-f382def7164d None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:32.048916 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 361/718] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Sep 19 16:00:31 2022] GET /identity/v3/services?name=network => generated 124 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:00:32.055163 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d98dbe-8ece-469d-84c7-25f830745860 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:32.225188 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d98dbe-8ece-469d-84c7-25f830745860 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:32.251132 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:32.274483 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b53771-579b-436c-8729-b810c6ac118a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:32.275209 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b53771-579b-436c-8729-b810c6ac118a None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:32.275556 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b53771-579b-436c-8729-b810c6ac118a None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:32.275898 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b53771-579b-436c-8729-b810c6ac118a None admin] PATH_INFO: `/v3/services` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:32.276622 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b53771-579b-436c-8729-b810c6ac118a None admin] RBAC: Adding query filter params (type=network) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:32.276999 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b53771-579b-436c-8729-b810c6ac118a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:32.279058 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b53771-579b-436c-8729-b810c6ac118a None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:32.291165 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 358/719] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Sep 19 16:00:32 2022] GET /identity/v3/services?type=network => generated 124 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:33.865015 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517732d5-000e-4227-9074-df826a31a62d None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:33.865522 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517732d5-000e-4227-9074-df826a31a62d None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:33.865598 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517732d5-000e-4227-9074-df826a31a62d None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:33.867226 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 362/720] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:00:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:00:33.872607 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f6f8c-fc0c-45a6-8157-011d73cb0bb0 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:33.873004 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f6f8c-fc0c-45a6-8157-011d73cb0bb0 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:33.873174 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f6f8c-fc0c-45a6-8157-011d73cb0bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:33.895683 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-019f6f8c-fc0c-45a6-8157-011d73cb0bb0 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:00:33.896879 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-019f6f8c-fc0c-45a6-8157-011d73cb0bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:00:33.984044 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 359/721] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:00:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 16:00:34.303312 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517732d5-000e-4227-9074-df826a31a62d None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:34.372686 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:34.438899 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517732d5-000e-4227-9074-df826a31a62d None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:34.441964 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e041997c-7b13-4ee3-ae24-a39af1b218c7 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:34.442289 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e041997c-7b13-4ee3-ae24-a39af1b218c7 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:34.442541 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e041997c-7b13-4ee3-ae24-a39af1b218c7 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:34.445645 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 360/722] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:00:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:00:34.450293 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2d93ba-a2c6-4f76-be4b-c7ec487dcef4 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:34.450624 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2d93ba-a2c6-4f76-be4b-c7ec487dcef4 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:34.450861 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2d93ba-a2c6-4f76-be4b-c7ec487dcef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:34.478743 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0889c296-b48b-4afa-931b-08091528258c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:34.479853 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0889c296-b48b-4afa-931b-08091528258c None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:34.480368 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0889c296-b48b-4afa-931b-08091528258c None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:34.480875 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0889c296-b48b-4afa-931b-08091528258c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:34.495033 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0889c296-b48b-4afa-931b-08091528258c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 16:00:34.501310 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-0889c296-b48b-4afa-931b-08091528258c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 16:00:34.501310 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:34.501310 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:34.501310 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:34.501310 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:34.501310 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:34.501310 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:34.501310 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:34.501310 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:34.501310 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:34.501310 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:34.501310 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:34.501310 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 16:00:34.501310 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 16:00:34.501310 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 16:00:34.501310 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 16:00:34.502298 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 16:00:34.502298 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 16:00:34.502298 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 16:00:34.502298 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 16:00:34.502298 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:34.502298 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:34.502298 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:34.502298 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:34.502298 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:34.502298 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:34.502298 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:34.502298 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:34.502298 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:34.502298 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:34.502298 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:34.502298 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:34.503181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:34.503181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:34.503181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:34.503181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:34.503181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 16:00:34.503181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 16:00:34.503181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 16:00:34.503181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 16:00:34.503181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 16:00:34.503181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 16:00:34.503181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 16:00:34.503181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:34.506126 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 363/723] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 19 16:00:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:34.511588 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0889c296-b48b-4afa-931b-08091528258c None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:34.640298 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-5e2d93ba-a2c6-4f76-be4b-c7ec487dcef4 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:00:34.641951 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-5e2d93ba-a2c6-4f76-be4b-c7ec487dcef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:00:34.673319 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0889c296-b48b-4afa-931b-08091528258c None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:34.689772 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:34.711778 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630cdff9-94d4-4cb6-9186-b22c15fd128f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:34.712391 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630cdff9-94d4-4cb6-9186-b22c15fd128f None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:34.712722 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630cdff9-94d4-4cb6-9186-b22c15fd128f None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:34.712998 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630cdff9-94d4-4cb6-9186-b22c15fd128f None admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:34.713667 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630cdff9-94d4-4cb6-9186-b22c15fd128f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:34.713986 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630cdff9-94d4-4cb6-9186-b22c15fd128f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:34.715699 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630cdff9-94d4-4cb6-9186-b22c15fd128f None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:34.725023 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 364/724] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 16:00:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:00:34.726274 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c626eb3c-32c4-43e3-af47-6515e9e5ff61 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:34.726591 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c626eb3c-32c4-43e3-af47-6515e9e5ff61 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:34.726869 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c626eb3c-32c4-43e3-af47-6515e9e5ff61 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:34.733303 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 365/725] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:00:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:00:34.734342 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c626eb3c-32c4-43e3-af47-6515e9e5ff61 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:34.735862 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 361/726] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:00:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 16:00:34.737357 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31c1c12-83be-4e9f-aa56-d5cbc77cdcd2 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:34.737879 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31c1c12-83be-4e9f-aa56-d5cbc77cdcd2 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:34.738363 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31c1c12-83be-4e9f-aa56-d5cbc77cdcd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:34.760324 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-a31c1c12-83be-4e9f-aa56-d5cbc77cdcd2 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:00:34.761858 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-a31c1c12-83be-4e9f-aa56-d5cbc77cdcd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:00:34.809274 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:34.854891 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 362/727] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:00:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 16:00:34.857123 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21297bd4-fd67-4c01-ba11-4e6e02ec8a67 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:34.857611 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21297bd4-fd67-4c01-ba11-4e6e02ec8a67 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:34.858125 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21297bd4-fd67-4c01-ba11-4e6e02ec8a67 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:34.859811 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 363/728] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:00:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:00:34.865680 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49983f0-e50e-4ba4-a3d5-eac04ed2394d None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:34.866223 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49983f0-e50e-4ba4-a3d5-eac04ed2394d None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:34.866720 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49983f0-e50e-4ba4-a3d5-eac04ed2394d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:34.879419 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c626eb3c-32c4-43e3-af47-6515e9e5ff61 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:34.907622 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-b49983f0-e50e-4ba4-a3d5-eac04ed2394d None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:00:34.909498 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-b49983f0-e50e-4ba4-a3d5-eac04ed2394d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:00:34.953437 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a30ca6-158f-4e9b-9db4-c9c6aec2d91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:34.954498 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a30ca6-158f-4e9b-9db4-c9c6aec2d91e None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:34.956107 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a30ca6-158f-4e9b-9db4-c9c6aec2d91e None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:34.956599 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a30ca6-158f-4e9b-9db4-c9c6aec2d91e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:34.977129 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a30ca6-158f-4e9b-9db4-c9c6aec2d91e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:34.979469 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a30ca6-158f-4e9b-9db4-c9c6aec2d91e None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:35.008711 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-e7a30ca6-158f-4e9b-9db4-c9c6aec2d91e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:35.008711 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:35.008711 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:35.008711 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:35.008711 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:35.008711 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:35.008711 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:35.008711 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:35.008711 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:35.008711 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:35.008711 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:35.008711 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:35.008711 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:00:35.008711 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:00:35.008711 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:00:35.008711 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:00:35.009684 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:35.009684 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:35.009684 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:35.009684 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:35.009684 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:35.009684 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:35.009684 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:35.009684 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:35.009684 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:35.009684 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:35.009684 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:35.009684 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:35.009684 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:35.009684 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:35.009684 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:35.009684 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:35.009684 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:00:35.010613 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:00:35.010613 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:35.010613 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:35.019998 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 366/729] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:00:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:35.022763 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a30ca6-158f-4e9b-9db4-c9c6aec2d91e None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:35.058366 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:35.058616 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 364/730] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:00:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 16:00:35.060218 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49983f0-e50e-4ba4-a3d5-eac04ed2394d None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:35.169174 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a30ca6-158f-4e9b-9db4-c9c6aec2d91e None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:35.204308 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a05150-d974-4b06-83bd-ad9d18c3fa92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:35.204923 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a05150-d974-4b06-83bd-ad9d18c3fa92 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:35.205260 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a05150-d974-4b06-83bd-ad9d18c3fa92 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:35.205534 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a05150-d974-4b06-83bd-ad9d18c3fa92 None admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:35.206181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a05150-d974-4b06-83bd-ad9d18c3fa92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:35.206565 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a05150-d974-4b06-83bd-ad9d18c3fa92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:35.207922 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a05150-d974-4b06-83bd-ad9d18c3fa92 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:35.214135 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49983f0-e50e-4ba4-a3d5-eac04ed2394d None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:35.215747 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:35.220818 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 367/731] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:00:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:35.221829 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a05150-d974-4b06-83bd-ad9d18c3fa92 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:35.252319 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8056c8-2786-452d-90e2-e2b6f2c0a3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:35.253198 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8056c8-2786-452d-90e2-e2b6f2c0a3af None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:35.253630 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8056c8-2786-452d-90e2-e2b6f2c0a3af None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:35.254211 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8056c8-2786-452d-90e2-e2b6f2c0a3af None admin] PATH_INFO: `/v3/roles/service` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:35.267828 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8056c8-2786-452d-90e2-e2b6f2c0a3af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 16:00:35.273918 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-da8056c8-2786-452d-90e2-e2b6f2c0a3af None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 16:00:35.273918 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:35.273918 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:35.273918 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:35.273918 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:35.273918 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:35.273918 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:35.273918 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:35.273918 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:35.273918 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:35.273918 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:35.273918 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:35.273918 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 16:00:35.273918 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 16:00:35.273918 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 16:00:35.273918 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 16:00:35.274896 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 16:00:35.274896 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 16:00:35.274896 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 16:00:35.274896 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 16:00:35.274896 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:35.274896 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:35.274896 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:35.274896 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:35.274896 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:35.274896 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:35.274896 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:35.274896 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:35.274896 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:35.274896 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:35.274896 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:35.274896 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:35.275730 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:35.275730 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:35.275730 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:35.275730 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:35.275730 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 16:00:35.275730 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 16:00:35.275730 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 16:00:35.275730 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 16:00:35.275730 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 16:00:35.275730 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 16:00:35.275730 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 16:00:35.275730 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:35.275730 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 365/732] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Sep 19 16:00:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:35.277267 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a70acd1-3c92-43f8-925f-690be34f657e None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:35.277550 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a70acd1-3c92-43f8-925f-690be34f657e None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:35.277823 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a70acd1-3c92-43f8-925f-690be34f657e None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:35.279293 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 366/733] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:00:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:00:35.283170 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a70acd1-3c92-43f8-925f-690be34f657e None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:35.326270 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:35.339503 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a05150-d974-4b06-83bd-ad9d18c3fa92 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:35.374485 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbeba72-3931-4196-b76c-e2548cbb6f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:35.375112 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbeba72-3931-4196-b76c-e2548cbb6f4d None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:35.375391 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbeba72-3931-4196-b76c-e2548cbb6f4d None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:35.375671 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbeba72-3931-4196-b76c-e2548cbb6f4d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:35.385500 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbeba72-3931-4196-b76c-e2548cbb6f4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 16:00:35.390827 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-5cbeba72-3931-4196-b76c-e2548cbb6f4d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 16:00:35.390827 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:35.390827 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:35.390827 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:35.390827 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:35.390827 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:35.390827 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:35.390827 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:35.390827 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:35.390827 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:35.390827 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:35.390827 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:35.390827 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 16:00:35.390827 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 16:00:35.390827 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 16:00:35.390827 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 16:00:35.391718 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 16:00:35.391718 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 16:00:35.391718 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 16:00:35.391718 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 16:00:35.391718 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:35.391718 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:35.391718 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:35.391718 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:35.391718 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:35.391718 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:35.391718 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:35.391718 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:35.391718 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:35.391718 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:35.391718 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:35.391718 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:35.392546 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:35.392546 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:35.392546 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:35.392546 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:35.392546 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 16:00:35.392546 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 16:00:35.392546 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 16:00:35.392546 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 16:00:35.392546 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 16:00:35.392546 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 16:00:35.392546 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 16:00:35.392546 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:35.393176 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a70acd1-3c92-43f8-925f-690be34f657e None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:35.393248 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 368/734] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 19 16:00:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:35.393248 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbeba72-3931-4196-b76c-e2548cbb6f4d None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:35.427935 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfa1dc2-54e3-4b42-bdd8-573d640db943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:35.428476 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa1dc2-54e3-4b42-bdd8-573d640db943 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:35.428753 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa1dc2-54e3-4b42-bdd8-573d640db943 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:35.429021 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa1dc2-54e3-4b42-bdd8-573d640db943 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:35.437082 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfa1dc2-54e3-4b42-bdd8-573d640db943 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:35.438541 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfa1dc2-54e3-4b42-bdd8-573d640db943 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:35.449381 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-dcfa1dc2-54e3-4b42-bdd8-573d640db943 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 19 16:00:35.449381 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:35.449381 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:35.449381 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:35.449381 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:35.449381 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:35.449381 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:35.449381 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:35.449381 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:35.449381 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:35.449381 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:35.449381 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:35.449381 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 16:00:35.449381 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 16:00:35.449381 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 16:00:35.449381 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 16:00:35.450401 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:35.450401 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:35.450401 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 16:00:35.450401 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:00:35.450401 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 16:00:35.450401 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:00:35.450401 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:35.450401 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:35.450401 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:35.450401 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:35.450401 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:35.450401 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:35.450401 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:35.450401 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:35.450401 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:35.450401 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:35.450401 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:35.450401 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:35.451346 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:35.451346 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:35.451346 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 16:00:35.451346 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 16:00:35.451346 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 16:00:35.451346 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 16:00:35.451346 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 16:00:35.451346 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 16:00:35.451346 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 19 16:00:35.451346 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:35.451346 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 367/735] 10.222.0.57 () {56 vars in 1093 bytes} [Mon Sep 19 16:00:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:35.451919 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec466bc-93ed-405e-bc2a-5c2b12b51b77 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:35.451919 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec466bc-93ed-405e-bc2a-5c2b12b51b77 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:35.452219 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec466bc-93ed-405e-bc2a-5c2b12b51b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:35.469909 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-6ec466bc-93ed-405e-bc2a-5c2b12b51b77 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:00:35.471146 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-6ec466bc-93ed-405e-bc2a-5c2b12b51b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:00:35.488803 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:35.496536 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbeba72-3931-4196-b76c-e2548cbb6f4d None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:35.531431 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae5f581-26f4-433a-92f8-6272fada4ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:35.532030 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae5f581-26f4-433a-92f8-6272fada4ed0 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:35.532308 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae5f581-26f4-433a-92f8-6272fada4ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:35.532583 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae5f581-26f4-433a-92f8-6272fada4ed0 None admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:35.533238 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae5f581-26f4-433a-92f8-6272fada4ed0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:35.533544 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae5f581-26f4-433a-92f8-6272fada4ed0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:35.534966 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae5f581-26f4-433a-92f8-6272fada4ed0 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:35.539095 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 368/736] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:00:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 16:00:35.539939 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec466bc-93ed-405e-bc2a-5c2b12b51b77 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:35.554753 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 369/737] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Sep 19 16:00:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:35.555759 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae5f581-26f4-433a-92f8-6272fada4ed0 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:35.577377 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:35.653988 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec466bc-93ed-405e-bc2a-5c2b12b51b77 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:35.674019 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae5f581-26f4-433a-92f8-6272fada4ed0 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:35.690860 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217aec93-4b0e-4bda-bc37-2ea00a516a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:35.691383 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217aec93-4b0e-4bda-bc37-2ea00a516a1f None admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:35.691657 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217aec93-4b0e-4bda-bc37-2ea00a516a1f None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:35.691922 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217aec93-4b0e-4bda-bc37-2ea00a516a1f None admin] PATH_INFO: `/v3/services` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:35.692567 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217aec93-4b0e-4bda-bc37-2ea00a516a1f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:35.693910 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217aec93-4b0e-4bda-bc37-2ea00a516a1f None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:35.710611 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 369/738] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Sep 19 16:00:35 2022] POST /identity/v3/services => generated 251 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:00:35.711352 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217aec93-4b0e-4bda-bc37-2ea00a516a1f None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:35.733731 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:35.740010 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f51e109-20d3-49cc-b884-45b63dcf8427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:35.741264 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f51e109-20d3-49cc-b884-45b63dcf8427 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:35.741721 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f51e109-20d3-49cc-b884-45b63dcf8427 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:35.742189 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f51e109-20d3-49cc-b884-45b63dcf8427 None admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:35.744576 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f51e109-20d3-49cc-b884-45b63dcf8427 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:35.745271 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f51e109-20d3-49cc-b884-45b63dcf8427 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:35.749855 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f51e109-20d3-49cc-b884-45b63dcf8427 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:35.770978 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 370/739] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 16:00:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:35.777529 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f51e109-20d3-49cc-b884-45b63dcf8427 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:35.871132 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217aec93-4b0e-4bda-bc37-2ea00a516a1f None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:35.884893 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:35.921046 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42500e48-2da4-4a1f-8acd-54c6d364bec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:35.922049 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42500e48-2da4-4a1f-8acd-54c6d364bec8 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:35.922332 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42500e48-2da4-4a1f-8acd-54c6d364bec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:35.923363 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42500e48-2da4-4a1f-8acd-54c6d364bec8 None admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:35.923486 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42500e48-2da4-4a1f-8acd-54c6d364bec8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:35.923712 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42500e48-2da4-4a1f-8acd-54c6d364bec8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:35.925161 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42500e48-2da4-4a1f-8acd-54c6d364bec8 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:35.928425 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f51e109-20d3-49cc-b884-45b63dcf8427 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:35.945446 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 370/740] 10.222.0.57 () {56 vars in 1133 bytes} [Mon Sep 19 16:00:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:35.945985 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42500e48-2da4-4a1f-8acd-54c6d364bec8 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:35.979223 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd1fddd-e150-4cd1-a3f7-ba8bf895b935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:35.980502 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd1fddd-e150-4cd1-a3f7-ba8bf895b935 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:35.981831 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd1fddd-e150-4cd1-a3f7-ba8bf895b935 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:35.982539 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd1fddd-e150-4cd1-a3f7-ba8bf895b935 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:35.998773 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd1fddd-e150-4cd1-a3f7-ba8bf895b935 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:36.001603 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd1fddd-e150-4cd1-a3f7-ba8bf895b935 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:36.027193 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-6dd1fddd-e150-4cd1-a3f7-ba8bf895b935 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:36.027193 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:36.027193 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:36.027193 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:36.027193 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:36.027193 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:36.027193 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:36.027193 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:36.027193 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:36.027193 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:36.027193 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:36.027193 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:36.027193 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:00:36.027193 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:00:36.027193 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:00:36.027193 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:00:36.028767 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:36.028767 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:36.028767 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:36.028767 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:36.028767 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:36.028767 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:36.028767 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:36.028767 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:36.028767 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:36.028767 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:36.028767 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:36.028767 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:36.028767 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:36.028767 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:36.028767 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:36.028767 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:36.028767 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:00:36.029793 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:00:36.029793 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:36.029793 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:36.029793 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 371/741] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:00:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:36.030542 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd1fddd-e150-4cd1-a3f7-ba8bf895b935 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:36.051974 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:36.088003 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42500e48-2da4-4a1f-8acd-54c6d364bec8 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:36.125234 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462a742d-aeec-48c5-98b9-d5691a320548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:36.126160 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462a742d-aeec-48c5-98b9-d5691a320548 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:36.126469 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462a742d-aeec-48c5-98b9-d5691a320548 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:36.126742 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462a742d-aeec-48c5-98b9-d5691a320548 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:36.138684 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462a742d-aeec-48c5-98b9-d5691a320548 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:36.140105 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462a742d-aeec-48c5-98b9-d5691a320548 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:36.140395 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd1fddd-e150-4cd1-a3f7-ba8bf895b935 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:36.156384 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-462a742d-aeec-48c5-98b9-d5691a320548 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:36.156384 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:36.156384 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:36.156384 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:36.156384 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:36.156384 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:36.156384 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:36.156384 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:36.156384 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:36.156384 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:36.156384 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:36.156384 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:36.156384 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:00:36.156384 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:00:36.156384 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:00:36.156384 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:00:36.157567 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:36.157567 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:36.157567 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:36.157567 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:36.157567 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:36.157567 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:36.157567 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:36.157567 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:36.157567 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:36.157567 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:36.157567 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:36.157567 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:36.157567 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:36.157567 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:36.157567 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:36.157567 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:36.157567 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:00:36.158615 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:00:36.158615 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:36.158615 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:36.158615 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 371/742] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:00:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:36.162925 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462a742d-aeec-48c5-98b9-d5691a320548 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:36.165158 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:36.180491 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8702e86f-eff4-4382-8bc0-43fcd4b49f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:36.181190 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8702e86f-eff4-4382-8bc0-43fcd4b49f8a None admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:36.181550 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8702e86f-eff4-4382-8bc0-43fcd4b49f8a None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:36.181892 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8702e86f-eff4-4382-8bc0-43fcd4b49f8a None admin] PATH_INFO: `/v3/services` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:36.182704 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8702e86f-eff4-4382-8bc0-43fcd4b49f8a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:36.184203 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8702e86f-eff4-4382-8bc0-43fcd4b49f8a None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:36.201005 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 372/743] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Sep 19 16:00:36 2022] POST /identity/v3/services => generated 240 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:00:36.202175 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8702e86f-eff4-4382-8bc0-43fcd4b49f8a None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:36.297497 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462a742d-aeec-48c5-98b9-d5691a320548 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:36.328450 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:36.345892 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8702e86f-eff4-4382-8bc0-43fcd4b49f8a None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:36.358357 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dede668-4c4b-4ed2-85d6-1880be6bfb4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:36.359043 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dede668-4c4b-4ed2-85d6-1880be6bfb4d None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:36.359315 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dede668-4c4b-4ed2-85d6-1880be6bfb4d None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:36.359582 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dede668-4c4b-4ed2-85d6-1880be6bfb4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:36.372520 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dede668-4c4b-4ed2-85d6-1880be6bfb4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:36.374313 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dede668-4c4b-4ed2-85d6-1880be6bfb4d None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:36.387756 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8983974a-d698-40e3-bdea-03542c7cb319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:36.388610 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8983974a-d698-40e3-bdea-03542c7cb319 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:36.388894 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8983974a-d698-40e3-bdea-03542c7cb319 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:36.389172 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8983974a-d698-40e3-bdea-03542c7cb319 None admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:36.389826 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8983974a-d698-40e3-bdea-03542c7cb319 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:36.390133 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8983974a-d698-40e3-bdea-03542c7cb319 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:36.391792 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8983974a-d698-40e3-bdea-03542c7cb319 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:36.391902 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-5dede668-4c4b-4ed2-85d6-1880be6bfb4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:36.391902 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:36.391902 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:36.391902 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:36.391902 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:36.391902 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:36.391902 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:36.391902 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:36.391902 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:36.391902 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:36.391902 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:36.391902 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:36.391902 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:00:36.391902 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:00:36.391902 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:00:36.391902 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:00:36.392788 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:36.392788 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:36.392788 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:36.392788 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:36.392788 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:36.392788 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:36.392788 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:36.392788 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:36.392788 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:36.392788 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:36.392788 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:36.392788 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:36.392788 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:36.392788 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:36.392788 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:36.392788 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:36.392788 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:00:36.393703 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:00:36.393703 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:36.393703 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:36.393703 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 372/744] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:00:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:36.394403 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dede668-4c4b-4ed2-85d6-1880be6bfb4d None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:36.405592 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 373/745] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:00:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:36.406308 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8983974a-d698-40e3-bdea-03542c7cb319 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:36.446540 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:36.522678 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dede668-4c4b-4ed2-85d6-1880be6bfb4d None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:36.539389 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8983974a-d698-40e3-bdea-03542c7cb319 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:36.565947 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0593138e-b407-4317-a648-18a44f58eedd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:36.566941 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593138e-b407-4317-a648-18a44f58eedd None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:36.567440 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593138e-b407-4317-a648-18a44f58eedd None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:36.567941 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593138e-b407-4317-a648-18a44f58eedd None admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:36.568782 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0593138e-b407-4317-a648-18a44f58eedd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:36.569291 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0593138e-b407-4317-a648-18a44f58eedd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:36.570887 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0593138e-b407-4317-a648-18a44f58eedd None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:36.582833 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db8bea1-0d43-43bd-883d-534023c5212f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:36.583439 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db8bea1-0d43-43bd-883d-534023c5212f None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:36.583774 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db8bea1-0d43-43bd-883d-534023c5212f None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:36.584052 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db8bea1-0d43-43bd-883d-534023c5212f None admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:36.584704 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db8bea1-0d43-43bd-883d-534023c5212f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:36.585011 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db8bea1-0d43-43bd-883d-534023c5212f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:36.585983 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 373/746] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:00:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:36.586396 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db8bea1-0d43-43bd-883d-534023c5212f None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:36.587256 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0593138e-b407-4317-a648-18a44f58eedd None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:36.600021 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 374/747] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:00:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:36.600825 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db8bea1-0d43-43bd-883d-534023c5212f None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:36.620209 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:36.701422 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0593138e-b407-4317-a648-18a44f58eedd None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:36.712516 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:36.719077 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db8bea1-0d43-43bd-883d-534023c5212f None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:36.741179 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fa0d04-3d97-4246-8ded-5c0b01d4295b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:36.742133 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa0d04-3d97-4246-8ded-5c0b01d4295b None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:36.742682 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa0d04-3d97-4246-8ded-5c0b01d4295b None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:36.743149 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa0d04-3d97-4246-8ded-5c0b01d4295b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:36.751730 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fa0d04-3d97-4246-8ded-5c0b01d4295b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:36.753553 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fa0d04-3d97-4246-8ded-5c0b01d4295b None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:36.769153 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-11fa0d04-3d97-4246-8ded-5c0b01d4295b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 16:00:36.769153 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:36.769153 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:36.769153 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:36.769153 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:36.769153 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:36.769153 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:36.769153 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:36.769153 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:36.769153 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:36.769153 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:36.769153 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:36.769153 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 16:00:36.769153 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 16:00:36.769153 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 16:00:36.769153 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 16:00:36.770328 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:36.770328 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:36.770328 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 16:00:36.770328 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:00:36.770328 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 16:00:36.770328 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:00:36.770328 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:36.770328 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:36.770328 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:36.770328 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:36.770328 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:36.770328 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:36.770328 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:36.770328 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:36.770328 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:36.770328 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:36.770328 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:36.770328 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:36.771542 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:36.771542 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:36.771542 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 16:00:36.771542 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 16:00:36.771542 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 16:00:36.771542 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 16:00:36.771542 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 16:00:36.771542 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 16:00:36.771542 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 16:00:36.771542 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:36.772536 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ceaa51-dcbd-4f24-ab78-dd4b7e3a3be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:36.773752 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ceaa51-dcbd-4f24-ab78-dd4b7e3a3be7 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:36.774405 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ceaa51-dcbd-4f24-ab78-dd4b7e3a3be7 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:36.774405 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ceaa51-dcbd-4f24-ab78-dd4b7e3a3be7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:36.776472 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 374/748] 10.222.0.57 () {56 vars in 1117 bytes} [Mon Sep 19 16:00:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:36.780482 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fa0d04-3d97-4246-8ded-5c0b01d4295b None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:36.786642 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ceaa51-dcbd-4f24-ab78-dd4b7e3a3be7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:36.788109 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ceaa51-dcbd-4f24-ab78-dd4b7e3a3be7 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:36.803143 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-46ceaa51-dcbd-4f24-ab78-dd4b7e3a3be7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 19 16:00:36.803143 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:36.803143 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:36.803143 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:36.803143 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:36.803143 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:36.803143 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:36.803143 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:36.803143 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:36.803143 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:36.803143 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:36.803143 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:36.803143 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 16:00:36.803143 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 16:00:36.803143 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 16:00:36.803143 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 16:00:36.804293 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:36.804293 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:36.804293 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 16:00:36.804293 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:00:36.804293 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 16:00:36.804293 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:00:36.804293 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:36.804293 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:36.804293 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:36.804293 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:36.804293 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:36.804293 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:36.804293 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:36.804293 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:36.804293 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:36.804293 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:36.804293 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:36.804293 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:36.805442 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:36.805442 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:36.805442 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 16:00:36.805442 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 16:00:36.805442 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 16:00:36.805442 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 16:00:36.805442 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 16:00:36.805442 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 16:00:36.805442 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 19 16:00:36.805442 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:36.805442 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 375/749] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 19 16:00:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:36.806141 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ceaa51-dcbd-4f24-ab78-dd4b7e3a3be7 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:36.904747 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:36.919684 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fa0d04-3d97-4246-8ded-5c0b01d4295b None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:36.933431 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ceaa51-dcbd-4f24-ab78-dd4b7e3a3be7 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:36.969856 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d07d5d1-0218-4572-94a3-636c0ccd792b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:36.972091 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d07d5d1-0218-4572-94a3-636c0ccd792b None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:36.972906 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d07d5d1-0218-4572-94a3-636c0ccd792b None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:36.973513 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d07d5d1-0218-4572-94a3-636c0ccd792b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:36.977088 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36244df5-4384-4def-bf11-de0eabe89015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:36.977720 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36244df5-4384-4def-bf11-de0eabe89015 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:36.977998 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36244df5-4384-4def-bf11-de0eabe89015 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:36.978281 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36244df5-4384-4def-bf11-de0eabe89015 None admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:36.979465 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36244df5-4384-4def-bf11-de0eabe89015 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:36.979774 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36244df5-4384-4def-bf11-de0eabe89015 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:36.981780 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36244df5-4384-4def-bf11-de0eabe89015 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:36.985938 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d07d5d1-0218-4572-94a3-636c0ccd792b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:36.987767 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d07d5d1-0218-4572-94a3-636c0ccd792b None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:37.003033 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 376/750] 10.222.0.57 () {56 vars in 1149 bytes} [Mon Sep 19 16:00:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:37.003740 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-4d07d5d1-0218-4572-94a3-636c0ccd792b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:00:37.003740 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:37.003740 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:37.003740 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:37.003740 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:37.003740 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:37.003740 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:37.003740 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:37.003740 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:37.003740 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:37.003740 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:37.003740 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:37.003740 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 16:00:37.003740 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 16:00:37.003740 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 16:00:37.004576 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 16:00:37.004576 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:37.004576 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:37.004576 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:37.004576 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:37.004576 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:37.004576 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:37.004576 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:37.004576 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:37.004576 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:37.004576 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:37.004576 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:37.004576 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:37.004576 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:37.004576 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:37.004576 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:37.004576 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:37.005452 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 16:00:37.005452 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 16:00:37.005452 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 16:00:37.005452 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 16:00:37.005452 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 16:00:37.005452 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 16:00:37.005452 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:00:37.005452 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:37.005872 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36244df5-4384-4def-bf11-de0eabe89015 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:37.007758 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 375/751] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 19 16:00:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:37.012925 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d07d5d1-0218-4572-94a3-636c0ccd792b None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:37.029974 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:37.135155 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36244df5-4384-4def-bf11-de0eabe89015 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:37.157784 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d07d5d1-0218-4572-94a3-636c0ccd792b None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:37.172566 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675f0524-efb5-488f-9b0d-6fca96a2d9ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:37.173173 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675f0524-efb5-488f-9b0d-6fca96a2d9ab None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:37.173449 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675f0524-efb5-488f-9b0d-6fca96a2d9ab None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:37.173719 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675f0524-efb5-488f-9b0d-6fca96a2d9ab None admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:37.174521 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675f0524-efb5-488f-9b0d-6fca96a2d9ab None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:37.174825 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675f0524-efb5-488f-9b0d-6fca96a2d9ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:37.176291 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675f0524-efb5-488f-9b0d-6fca96a2d9ab None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:37.196914 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 377/752] 10.222.0.57 () {56 vars in 1135 bytes} [Mon Sep 19 16:00:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:37.197462 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675f0524-efb5-488f-9b0d-6fca96a2d9ab None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:37.208330 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb9993d-a3ac-4b2e-b96d-5a9fe7a271ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:37.209762 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb9993d-a3ac-4b2e-b96d-5a9fe7a271ec None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:37.210276 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb9993d-a3ac-4b2e-b96d-5a9fe7a271ec None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:37.210838 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb9993d-a3ac-4b2e-b96d-5a9fe7a271ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:37.234529 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:37.271920 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:37.298671 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb9993d-a3ac-4b2e-b96d-5a9fe7a271ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:37.299797 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb9993d-a3ac-4b2e-b96d-5a9fe7a271ec None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:37.314077 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675f0524-efb5-488f-9b0d-6fca96a2d9ab None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:37.351691 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e578e7a-511d-4ef4-9c9f-01d218926fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:37.352297 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e578e7a-511d-4ef4-9c9f-01d218926fab None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:37.352572 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e578e7a-511d-4ef4-9c9f-01d218926fab None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:37.352841 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e578e7a-511d-4ef4-9c9f-01d218926fab None admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:37.353524 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e578e7a-511d-4ef4-9c9f-01d218926fab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:37.353825 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e578e7a-511d-4ef4-9c9f-01d218926fab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:37.355310 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e578e7a-511d-4ef4-9c9f-01d218926fab None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:37.368376 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 378/753] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 19 16:00:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:37.374053 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e578e7a-511d-4ef4-9c9f-01d218926fab None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:37.431416 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 376/754] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 19 16:00:37 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 16:00:37.432700 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb9993d-a3ac-4b2e-b96d-5a9fe7a271ec None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:37.491249 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:37.553946 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb9993d-a3ac-4b2e-b96d-5a9fe7a271ec None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:37.596758 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e578e7a-511d-4ef4-9c9f-01d218926fab None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:37.598598 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0400d8af-d101-4192-a736-ad5074d1750a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:37.600462 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0400d8af-d101-4192-a736-ad5074d1750a None admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:37.600954 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0400d8af-d101-4192-a736-ad5074d1750a None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:37.601456 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0400d8af-d101-4192-a736-ad5074d1750a None admin] PATH_INFO: `/v3/projects/be03e5300b3343169f35c643c93d1e06/users/1ecd01cc72834b72b5ff2c4b47249737/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:37.603653 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0400d8af-d101-4192-a736-ad5074d1750a None admin] RBAC: Authorizing `identity:create_grant(project_id=be03e5300b3343169f35c643c93d1e06, user_id=1ecd01cc72834b72b5ff2c4b47249737, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:37.605979 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0400d8af-d101-4192-a736-ad5074d1750a None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:37.632173 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 377/755] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Sep 19 16:00:37 2022] PUT /identity/v3/projects/be03e5300b3343169f35c643c93d1e06/users/1ecd01cc72834b72b5ff2c4b47249737/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:00:37.632173 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0400d8af-d101-4192-a736-ad5074d1750a None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:37.641403 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:37.653290 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e197a384-0c63-4d17-9267-0bbd020b4b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:37.653913 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e197a384-0c63-4d17-9267-0bbd020b4b2e None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:37.654234 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e197a384-0c63-4d17-9267-0bbd020b4b2e None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:37.654531 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e197a384-0c63-4d17-9267-0bbd020b4b2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:37.793594 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e197a384-0c63-4d17-9267-0bbd020b4b2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:37.795701 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e197a384-0c63-4d17-9267-0bbd020b4b2e None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:37.882219 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0400d8af-d101-4192-a736-ad5074d1750a None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:37.908125 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:37.945779 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0b262b-9281-4de7-bcae-71436add11f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:37.946925 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 379/756] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 19 16:00:37 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 578 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 16:00:37.947594 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b262b-9281-4de7-bcae-71436add11f8 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:37.948098 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b262b-9281-4de7-bcae-71436add11f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:37.948583 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b262b-9281-4de7-bcae-71436add11f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:37.953503 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e197a384-0c63-4d17-9267-0bbd020b4b2e None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:37.964275 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0b262b-9281-4de7-bcae-71436add11f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:37.967566 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0b262b-9281-4de7-bcae-71436add11f8 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:37.993095 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-1b0b262b-9281-4de7-bcae-71436add11f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:37.993095 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:37.993095 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:37.993095 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:37.993095 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:37.993095 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:37.993095 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:37.993095 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:37.993095 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:37.993095 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:37.993095 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:37.993095 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:37.993095 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:00:37.993095 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:00:37.993095 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:00:37.993095 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:00:37.994546 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:37.994546 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:37.994546 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:37.994546 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:37.994546 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:37.994546 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:37.994546 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:37.994546 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:37.994546 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:37.994546 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:37.994546 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:37.994546 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:37.994546 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:37.994546 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:37.994546 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:37.994546 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:37.994546 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:00:37.995403 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:00:37.995403 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:37.995403 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:37.995403 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 378/757] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:00:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:38.006763 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0b262b-9281-4de7-bcae-71436add11f8 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:38.010708 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:38.105727 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e197a384-0c63-4d17-9267-0bbd020b4b2e None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:38.125528 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0b262b-9281-4de7-bcae-71436add11f8 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:38.148038 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc10c63d-5e81-4579-9c9b-f4266e1fda4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:38.149033 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc10c63d-5e81-4579-9c9b-f4266e1fda4b None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:38.149506 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc10c63d-5e81-4579-9c9b-f4266e1fda4b None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:38.150065 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc10c63d-5e81-4579-9c9b-f4266e1fda4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:38.162670 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc10c63d-5e81-4579-9c9b-f4266e1fda4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:38.163492 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e292fc7-143a-4b3c-8d2e-ca7fbd837d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:38.164167 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc10c63d-5e81-4579-9c9b-f4266e1fda4b None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:38.164625 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e292fc7-143a-4b3c-8d2e-ca7fbd837d02 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:38.165148 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e292fc7-143a-4b3c-8d2e-ca7fbd837d02 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:38.165603 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e292fc7-143a-4b3c-8d2e-ca7fbd837d02 None admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:38.167035 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e292fc7-143a-4b3c-8d2e-ca7fbd837d02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:38.167978 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e292fc7-143a-4b3c-8d2e-ca7fbd837d02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:38.169798 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e292fc7-143a-4b3c-8d2e-ca7fbd837d02 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:38.183122 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-bc10c63d-5e81-4579-9c9b-f4266e1fda4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:38.183122 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:38.183122 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:38.183122 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:38.183122 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:38.183122 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:38.183122 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:38.183122 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:38.183122 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:38.183122 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:38.183122 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:38.183122 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:38.183122 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:00:38.183122 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:00:38.183122 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:00:38.183122 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:00:38.185021 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:38.185021 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:38.185021 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:38.185021 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:38.185021 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:38.185021 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:38.185021 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:38.185021 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:38.185021 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:38.185021 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:38.185021 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:38.185021 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:38.185021 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:38.185021 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:38.185021 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:38.185021 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:38.185021 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:00:38.186228 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:00:38.186228 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:38.186228 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:38.189673 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 380/759] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:00:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:38.189945 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 379/759] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:00:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:38.191187 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2d53d9-8fbe-4976-a9c0-47a93a8e3d28 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:38.191427 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e292fc7-143a-4b3c-8d2e-ca7fbd837d02 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:38.191529 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2d53d9-8fbe-4976-a9c0-47a93a8e3d28 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:38.191877 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2d53d9-8fbe-4976-a9c0-47a93a8e3d28 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:38.197304 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 381/760] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:00:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:00:38.197304 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2d53d9-8fbe-4976-a9c0-47a93a8e3d28 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:38.228278 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:38.301444 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:38.322253 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2d53d9-8fbe-4976-a9c0-47a93a8e3d28 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:38.360402 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e292fc7-143a-4b3c-8d2e-ca7fbd837d02 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:38.364482 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fd7e23-abca-4e4e-8c53-8684c2678009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:38.365362 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd7e23-abca-4e4e-8c53-8684c2678009 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:38.365843 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd7e23-abca-4e4e-8c53-8684c2678009 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:38.366300 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd7e23-abca-4e4e-8c53-8684c2678009 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:38.374386 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fd7e23-abca-4e4e-8c53-8684c2678009 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:38.376107 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fd7e23-abca-4e4e-8c53-8684c2678009 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:38.389247 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-c2fd7e23-abca-4e4e-8c53-8684c2678009 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:00:38.389247 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:38.389247 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:38.389247 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:38.389247 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:38.389247 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:38.389247 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:38.389247 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:38.389247 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:38.389247 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:38.389247 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:38.389247 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:38.389247 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 16:00:38.389247 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 16:00:38.389247 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 16:00:38.390223 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 16:00:38.390223 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:38.390223 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:38.390223 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:38.390223 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:38.390223 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:38.390223 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:38.390223 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:38.390223 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:38.390223 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:38.390223 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:38.390223 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:38.390223 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:38.390223 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:38.390223 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:38.390223 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:38.390223 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:38.391233 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 16:00:38.391233 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 16:00:38.391233 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 16:00:38.391233 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 16:00:38.391233 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 16:00:38.391233 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 16:00:38.391233 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:00:38.391233 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:38.393393 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 382/761] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 19 16:00:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:38.395137 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5353ff9-00df-4cf8-97d4-7f9c5417b29d None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:38.395608 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5353ff9-00df-4cf8-97d4-7f9c5417b29d None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:38.396073 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5353ff9-00df-4cf8-97d4-7f9c5417b29d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:38.416631 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-e5353ff9-00df-4cf8-97d4-7f9c5417b29d None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:00:38.418109 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-e5353ff9-00df-4cf8-97d4-7f9c5417b29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:00:38.423270 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38202c5f-9ad2-4c54-a910-7891bd360236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:38.423519 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38202c5f-9ad2-4c54-a910-7891bd360236 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:38.424015 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38202c5f-9ad2-4c54-a910-7891bd360236 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:38.424499 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38202c5f-9ad2-4c54-a910-7891bd360236 None admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:38.425803 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38202c5f-9ad2-4c54-a910-7891bd360236 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:38.426364 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38202c5f-9ad2-4c54-a910-7891bd360236 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:38.428879 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38202c5f-9ad2-4c54-a910-7891bd360236 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:38.458484 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 380/762] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:00:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:38.459966 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38202c5f-9ad2-4c54-a910-7891bd360236 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:38.520486 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 383/763] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:00:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 16:00:38.521711 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5353ff9-00df-4cf8-97d4-7f9c5417b29d None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:38.589681 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:38.610247 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38202c5f-9ad2-4c54-a910-7891bd360236 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:38.620736 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:38.646809 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3012f5-0a8a-4018-acbb-3dbbbb9e88a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:38.647521 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3012f5-0a8a-4018-acbb-3dbbbb9e88a7 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:38.647695 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3012f5-0a8a-4018-acbb-3dbbbb9e88a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:38.647972 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3012f5-0a8a-4018-acbb-3dbbbb9e88a7 None admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:38.648660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3012f5-0a8a-4018-acbb-3dbbbb9e88a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:38.649202 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3012f5-0a8a-4018-acbb-3dbbbb9e88a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:38.650404 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3012f5-0a8a-4018-acbb-3dbbbb9e88a7 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:38.656375 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5353ff9-00df-4cf8-97d4-7f9c5417b29d None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:38.668794 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 381/764] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 19 16:00:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:38.670080 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d5dd5-2dce-4f40-aa05-02a5ac9c5138 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:38.670391 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d5dd5-2dce-4f40-aa05-02a5ac9c5138 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:38.670669 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d5dd5-2dce-4f40-aa05-02a5ac9c5138 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:38.678071 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 382/765] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:00:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:00:38.679836 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93d5dd5-2dce-4f40-aa05-02a5ac9c5138 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:38.700516 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154b2436-dbb4-4311-9eb8-f8bcbf723171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:38.701448 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b2436-dbb4-4311-9eb8-f8bcbf723171 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:38.701899 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b2436-dbb4-4311-9eb8-f8bcbf723171 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:38.702343 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b2436-dbb4-4311-9eb8-f8bcbf723171 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:38.712955 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154b2436-dbb4-4311-9eb8-f8bcbf723171 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:38.716068 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154b2436-dbb4-4311-9eb8-f8bcbf723171 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:38.735555 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-154b2436-dbb4-4311-9eb8-f8bcbf723171 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:00:38.735555 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:38.735555 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:38.735555 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:38.735555 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:38.735555 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:38.735555 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:38.735555 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:38.735555 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:38.735555 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:38.735555 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:38.735555 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:38.735555 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 16:00:38.735555 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 16:00:38.735555 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 16:00:38.736494 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 16:00:38.736494 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:38.736494 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:38.736494 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:38.736494 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:38.736494 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:38.736494 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:38.736494 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:38.736494 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:38.736494 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:38.736494 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:38.736494 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:38.736494 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:38.736494 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:38.736494 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:38.736494 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:38.736494 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:38.737402 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 16:00:38.737402 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 16:00:38.737402 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 16:00:38.737402 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 16:00:38.737402 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 16:00:38.737402 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 16:00:38.737402 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:00:38.737402 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:38.739809 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 384/766] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 19 16:00:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:38.741842 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f4e0b-4cf5-4ce6-af77-17c48573b165 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:38.742278 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f4e0b-4cf5-4ce6-af77-17c48573b165 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:38.742717 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f4e0b-4cf5-4ce6-af77-17c48573b165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:38.768771 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-d57f4e0b-4cf5-4ce6-af77-17c48573b165 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:00:38.770888 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-d57f4e0b-4cf5-4ce6-af77-17c48573b165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:00:38.799995 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93d5dd5-2dce-4f40-aa05-02a5ac9c5138 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:38.839005 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a580bcc7-655e-4442-b010-d656f49f2e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:38.839919 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a580bcc7-655e-4442-b010-d656f49f2e4a None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:38.840200 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a580bcc7-655e-4442-b010-d656f49f2e4a None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:38.840501 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a580bcc7-655e-4442-b010-d656f49f2e4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:38.841274 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a580bcc7-655e-4442-b010-d656f49f2e4a None admin] RBAC: Adding query filter params (role.id=1e475e6a170a4dbabc4ce8348eeb30b9, scope.project.id=be03e5300b3343169f35c643c93d1e06, user.id=7222bd3b313b47e5a4fa0ebc9ff0cdd5) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:38.841588 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a580bcc7-655e-4442-b010-d656f49f2e4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:38.843803 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a580bcc7-655e-4442-b010-d656f49f2e4a None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:38.858414 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 383/767] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 19 16:00:38 2022] GET /identity/v3/role_assignments?user.id=7222bd3b313b47e5a4fa0ebc9ff0cdd5&scope.project.id=be03e5300b3343169f35c643c93d1e06&role.id=1e475e6a170a4dbabc4ce8348eeb30b9 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:38.858414 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a580bcc7-655e-4442-b010-d656f49f2e4a None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:38.865795 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 385/768] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:00:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 16:00:38.866871 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57f4e0b-4cf5-4ce6-af77-17c48573b165 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:38.867989 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:38.952613 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:39.007464 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a580bcc7-655e-4442-b010-d656f49f2e4a None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:39.040151 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57f4e0b-4cf5-4ce6-af77-17c48573b165 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:39.084853 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1f309f-7204-44e7-9622-22905475de40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:39.085863 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1f309f-7204-44e7-9622-22905475de40 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:39.086564 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1f309f-7204-44e7-9622-22905475de40 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:39.088453 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1f309f-7204-44e7-9622-22905475de40 None admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:39.089470 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1f309f-7204-44e7-9622-22905475de40 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:39.090233 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1f309f-7204-44e7-9622-22905475de40 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:39.093525 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1f309f-7204-44e7-9622-22905475de40 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:39.113385 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 384/769] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 19 16:00:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:39.117864 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7555272-5a39-4704-8bac-363e3c9bb025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:39.118852 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7555272-5a39-4704-8bac-363e3c9bb025 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:39.119969 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1f309f-7204-44e7-9622-22905475de40 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:39.120894 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7555272-5a39-4704-8bac-363e3c9bb025 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:39.121168 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7555272-5a39-4704-8bac-363e3c9bb025 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:39.121797 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7555272-5a39-4704-8bac-363e3c9bb025 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:39.124572 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7555272-5a39-4704-8bac-363e3c9bb025 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:39.143444 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-c7555272-5a39-4704-8bac-363e3c9bb025 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 19 16:00:39.143444 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:39.143444 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 16:00:39.143444 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 16:00:39.143444 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 16:00:39.143444 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 16:00:39.143444 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 16:00:39.143444 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 16:00:39.143444 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 19 16:00:39.143444 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:39.143444 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 16:00:39.143444 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:39.143444 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:39.143444 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:39.143444 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:39.143444 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:39.143444 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:39.144498 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:39.144498 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:39.144498 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:39.144498 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:39.144498 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:39.144498 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:39.144498 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 16:00:39.144498 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 16:00:39.144498 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 16:00:39.144498 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 16:00:39.144498 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:39.144498 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:39.144498 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:39.144498 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:39.144498 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:39.144498 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:39.145317 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:39.145317 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:39.145317 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:39.145317 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:39.145317 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:39.145317 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:39.145317 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:39.145317 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:39.145317 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:39.145317 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:39.145317 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 16:00:39.145317 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 16:00:39.145317 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 19 16:00:39.145317 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:39.145317 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 386/770] 10.222.0.57 () {56 vars in 1129 bytes} [Mon Sep 19 16:00:38 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:39.154631 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7555272-5a39-4704-8bac-363e3c9bb025 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:39.212856 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:39.257409 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1f309f-7204-44e7-9622-22905475de40 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:39.276856 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7555272-5a39-4704-8bac-363e3c9bb025 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:39.299369 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825d4550-a009-474b-97de-cefece6535d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:39.300222 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825d4550-a009-474b-97de-cefece6535d3 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:39.300673 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825d4550-a009-474b-97de-cefece6535d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:39.301111 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825d4550-a009-474b-97de-cefece6535d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:39.302459 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825d4550-a009-474b-97de-cefece6535d3 None admin] RBAC: Adding query filter params (role.id=63d2d197bfd6497583e94bfb065ce29b, scope.project.id=be03e5300b3343169f35c643c93d1e06, user.id=884a0524aca744b99c7496c11c3291ae) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:39.302924 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825d4550-a009-474b-97de-cefece6535d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:39.304837 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825d4550-a009-474b-97de-cefece6535d3 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:39.309575 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:39.312286 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 385/771] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 19 16:00:39 2022] GET /identity/v3/role_assignments?user.id=884a0524aca744b99c7496c11c3291ae&scope.project.id=be03e5300b3343169f35c643c93d1e06&role.id=63d2d197bfd6497583e94bfb065ce29b => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:39.313358 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825d4550-a009-474b-97de-cefece6535d3 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:39.320937 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d9a2a-4c77-4971-97f5-8a54b82c6702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:39.321753 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d9a2a-4c77-4971-97f5-8a54b82c6702 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:39.322127 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d9a2a-4c77-4971-97f5-8a54b82c6702 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:39.322552 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d9a2a-4c77-4971-97f5-8a54b82c6702 None admin] PATH_INFO: `/v3/services` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:39.323294 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00d9a2a-4c77-4971-97f5-8a54b82c6702 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:39.323697 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00d9a2a-4c77-4971-97f5-8a54b82c6702 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:39.326323 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00d9a2a-4c77-4971-97f5-8a54b82c6702 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:39.332954 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 387/772] 10.222.0.57 () {56 vars in 1124 bytes} [Mon Sep 19 16:00:39 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:39.338830 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d9a2a-4c77-4971-97f5-8a54b82c6702 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:39.534862 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825d4550-a009-474b-97de-cefece6535d3 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:39.565306 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d9a2a-4c77-4971-97f5-8a54b82c6702 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:39.569058 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:39.579441 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab28870d-4ef4-4a62-ab7f-f86ebb50c41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:39.580082 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab28870d-4ef4-4a62-ab7f-f86ebb50c41f None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:39.580361 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab28870d-4ef4-4a62-ab7f-f86ebb50c41f None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:39.580633 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab28870d-4ef4-4a62-ab7f-f86ebb50c41f None admin] PATH_INFO: `/v3/services/network` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:39.581531 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab28870d-4ef4-4a62-ab7f-f86ebb50c41f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:39.582631 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab28870d-4ef4-4a62-ab7f-f86ebb50c41f None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:39.596084 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-ab28870d-4ef4-4a62-ab7f-f86ebb50c41f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 19 16:00:39.596084 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:39.596084 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 16:00:39.596084 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 16:00:39.596084 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 16:00:39.596084 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 16:00:39.596084 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 16:00:39.596084 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 16:00:39.596084 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 19 16:00:39.596084 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:39.596084 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 16:00:39.596084 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:39.596084 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:39.596084 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:39.596084 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:39.596084 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:39.596084 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:39.597100 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:39.597100 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:39.597100 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:39.597100 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:39.597100 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:39.597100 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:39.597100 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 16:00:39.597100 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 16:00:39.597100 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 16:00:39.597100 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 16:00:39.597100 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:39.597100 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:39.597100 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:39.597100 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:39.597100 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:39.597100 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:39.597913 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:39.597913 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:39.597913 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:39.597913 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:39.597913 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:39.597913 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:39.597913 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:39.597913 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:39.597913 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:39.597913 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:39.597913 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 16:00:39.597913 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 16:00:39.597913 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 19 16:00:39.597913 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:39.597913 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 386/773] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 19 16:00:39 2022] GET /identity/v3/services/network => generated 88 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:39.608692 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab28870d-4ef4-4a62-ab7f-f86ebb50c41f None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:39.618866 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a7b9a2-5bc4-44c4-ac81-dffcb7345a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:39.619870 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a7b9a2-5bc4-44c4-ac81-dffcb7345a6d None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:39.620407 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a7b9a2-5bc4-44c4-ac81-dffcb7345a6d None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:39.620872 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a7b9a2-5bc4-44c4-ac81-dffcb7345a6d None admin] PATH_INFO: `/v3/services` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:39.622222 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a7b9a2-5bc4-44c4-ac81-dffcb7345a6d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:39.622893 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a7b9a2-5bc4-44c4-ac81-dffcb7345a6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:39.624568 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a7b9a2-5bc4-44c4-ac81-dffcb7345a6d None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:39.631804 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 388/774] 10.222.0.57 () {56 vars in 1124 bytes} [Mon Sep 19 16:00:39 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:00:39.642426 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a7b9a2-5bc4-44c4-ac81-dffcb7345a6d None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:39.766341 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab28870d-4ef4-4a62-ab7f-f86ebb50c41f None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:39.778854 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:39.811465 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadec518-fdac-4077-87df-a5533fdd588d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:39.812430 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadec518-fdac-4077-87df-a5533fdd588d None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:39.812844 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadec518-fdac-4077-87df-a5533fdd588d None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:39.813239 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadec518-fdac-4077-87df-a5533fdd588d None admin] PATH_INFO: `/v3/services` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:39.814028 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadec518-fdac-4077-87df-a5533fdd588d None admin] RBAC: Adding query filter params (name=network) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:39.814528 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadec518-fdac-4077-87df-a5533fdd588d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:39.816030 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadec518-fdac-4077-87df-a5533fdd588d None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:39.821560 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 387/775] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Sep 19 16:00:39 2022] GET /identity/v3/services?name=network => generated 124 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:39.827469 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadec518-fdac-4077-87df-a5533fdd588d None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:39.895707 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:39.924456 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a7b9a2-5bc4-44c4-ac81-dffcb7345a6d None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:39.943476 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadec518-fdac-4077-87df-a5533fdd588d None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:39.983220 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37bbce4-c3a4-4c51-819d-fedccab3762b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:39.984145 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37bbce4-c3a4-4c51-819d-fedccab3762b None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:39.984592 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37bbce4-c3a4-4c51-819d-fedccab3762b None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:39.985015 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37bbce4-c3a4-4c51-819d-fedccab3762b None admin] PATH_INFO: `/v3/services` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:39.985823 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37bbce4-c3a4-4c51-819d-fedccab3762b None admin] RBAC: Adding query filter params (type=network) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:39.986649 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37bbce4-c3a4-4c51-819d-fedccab3762b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:39.989295 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37bbce4-c3a4-4c51-819d-fedccab3762b None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:39.996164 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 388/776] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Sep 19 16:00:39 2022] GET /identity/v3/services?type=network => generated 350 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:40.003986 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37bbce4-c3a4-4c51-819d-fedccab3762b None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:40.030872 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c1b6f2-42bb-4ef1-88fc-44e656327abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:40.031415 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1b6f2-42bb-4ef1-88fc-44e656327abf None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:40.031494 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1b6f2-42bb-4ef1-88fc-44e656327abf None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:40.031770 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1b6f2-42bb-4ef1-88fc-44e656327abf None admin] PATH_INFO: `/v3/endpoints` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:40.032490 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c1b6f2-42bb-4ef1-88fc-44e656327abf None admin] RBAC: Adding query filter params (interface=public, service_id=4c7127774cb2486ea52d65ce48eb807b, region_id=RegionOne) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:40.032801 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c1b6f2-42bb-4ef1-88fc-44e656327abf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:40.034176 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c1b6f2-42bb-4ef1-88fc-44e656327abf None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:40.056343 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 389/777] 10.222.0.57 () {56 vars in 1251 bytes} [Mon Sep 19 16:00:39 2022] GET /identity/v3/endpoints?service_id=4c7127774cb2486ea52d65ce48eb807b&interface=public®ion_id=RegionOne => generated 194 bytes in 416 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:40.071624 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c1b6f2-42bb-4ef1-88fc-44e656327abf None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:40.126632 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37bbce4-c3a4-4c51-819d-fedccab3762b None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:40.153395 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:40.173869 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2a6da6-dc52-472d-9887-0c2499a6e93a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:40.175126 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a6da6-dc52-472d-9887-0c2499a6e93a None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:40.175467 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a6da6-dc52-472d-9887-0c2499a6e93a None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:40.175821 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a6da6-dc52-472d-9887-0c2499a6e93a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:40.176886 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2a6da6-dc52-472d-9887-0c2499a6e93a None admin] RBAC: Adding query filter params (interface=public, service_id=6dee865a96154947a71d99c07a0e641d, region_id=RegionOne) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:40.177268 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2a6da6-dc52-472d-9887-0c2499a6e93a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:40.178985 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2a6da6-dc52-472d-9887-0c2499a6e93a None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:40.189636 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 389/778] 10.222.0.57 () {56 vars in 1251 bytes} [Mon Sep 19 16:00:40 2022] GET /identity/v3/endpoints?service_id=6dee865a96154947a71d99c07a0e641d&interface=public®ion_id=RegionOne => generated 194 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:00:40.194412 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b04883c-0b34-45cc-97bb-77967fc46dc5 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:40.194412 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b04883c-0b34-45cc-97bb-77967fc46dc5 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:40.194412 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b04883c-0b34-45cc-97bb-77967fc46dc5 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:40.197329 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 390/779] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:00:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:00:40.199362 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b04883c-0b34-45cc-97bb-77967fc46dc5 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:40.230313 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c1b6f2-42bb-4ef1-88fc-44e656327abf None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:40.254481 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:40.270584 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e298da-83bf-4768-ade6-5bc1d8cbc919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:40.271497 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e298da-83bf-4768-ade6-5bc1d8cbc919 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:40.271976 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e298da-83bf-4768-ade6-5bc1d8cbc919 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:40.272442 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e298da-83bf-4768-ade6-5bc1d8cbc919 None admin] PATH_INFO: `/v3/services` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:40.273227 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e298da-83bf-4768-ade6-5bc1d8cbc919 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:40.274875 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e298da-83bf-4768-ade6-5bc1d8cbc919 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:40.281429 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 390/780] 10.222.0.57 () {56 vars in 1087 bytes} [Mon Sep 19 16:00:40 2022] GET /identity/v3/services => generated 772 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:40.287991 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb5626b-5dc9-467a-b313-98956cc48696 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:40.288613 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb5626b-5dc9-467a-b313-98956cc48696 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:40.289085 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb5626b-5dc9-467a-b313-98956cc48696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:40.324792 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-7bb5626b-5dc9-467a-b313-98956cc48696 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:00:40.326340 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-7bb5626b-5dc9-467a-b313-98956cc48696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:00:40.331627 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b04883c-0b34-45cc-97bb-77967fc46dc5 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:40.382440 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f33846c-be56-43c0-81ab-5947833124b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:40.382440 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f33846c-be56-43c0-81ab-5947833124b0 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:40.382440 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f33846c-be56-43c0-81ab-5947833124b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:40.382440 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f33846c-be56-43c0-81ab-5947833124b0 None admin] PATH_INFO: `/v3/services` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:40.382831 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f33846c-be56-43c0-81ab-5947833124b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:40.382831 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f33846c-be56-43c0-81ab-5947833124b0 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:40.388566 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 391/781] 10.222.0.57 () {56 vars in 1087 bytes} [Mon Sep 19 16:00:40 2022] GET /identity/v3/services => generated 772 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:40.456798 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 391/782] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:00:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 16:00:40.804494 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f33846c-be56-43c0-81ab-5947833124b0 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:40.875055 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:40.939301 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f33846c-be56-43c0-81ab-5947833124b0 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:40.982602 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5d4cd9-9f3b-4c59-b090-0466935b1ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:40.983263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d4cd9-9f3b-4c59-b090-0466935b1ff0 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:40.983544 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d4cd9-9f3b-4c59-b090-0466935b1ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:40.983819 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d4cd9-9f3b-4c59-b090-0466935b1ff0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:40.995640 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5d4cd9-9f3b-4c59-b090-0466935b1ff0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 16:00:41.005842 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-5e5d4cd9-9f3b-4c59-b090-0466935b1ff0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 16:00:41.005842 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:41.005842 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:41.005842 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:41.005842 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:41.005842 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:41.005842 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:41.005842 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:41.005842 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:41.005842 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:41.005842 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:41.005842 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:41.005842 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 16:00:41.005842 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 16:00:41.005842 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 16:00:41.005842 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 16:00:41.007014 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 16:00:41.007014 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 16:00:41.007014 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 16:00:41.007014 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 16:00:41.007014 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:41.007014 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:41.007014 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:41.007014 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:41.007014 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:41.007014 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:41.007014 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:41.007014 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:41.007014 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:41.007014 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:41.007014 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:41.007014 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:41.007986 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:41.007986 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:41.007986 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:41.007986 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:41.007986 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 16:00:41.007986 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 16:00:41.007986 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 16:00:41.007986 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 16:00:41.007986 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 16:00:41.007986 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 16:00:41.007986 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 16:00:41.007986 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:41.007986 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 392/783] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 19 16:00:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:41.014742 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb5626b-5dc9-467a-b313-98956cc48696 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:41.167511 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb5626b-5dc9-467a-b313-98956cc48696 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:41.192048 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:41.217571 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3fa48b-7f36-4211-a205-333e79bc5582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:41.218186 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3fa48b-7f36-4211-a205-333e79bc5582 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:41.218489 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3fa48b-7f36-4211-a205-333e79bc5582 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:41.218764 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3fa48b-7f36-4211-a205-333e79bc5582 None admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:41.219432 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3fa48b-7f36-4211-a205-333e79bc5582 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:41.219735 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3fa48b-7f36-4211-a205-333e79bc5582 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:41.221099 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3fa48b-7f36-4211-a205-333e79bc5582 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:41.234484 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 392/784] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 16:00:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:41.240287 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5d4cd9-9f3b-4c59-b090-0466935b1ff0 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:41.376185 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5d4cd9-9f3b-4c59-b090-0466935b1ff0 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:41.394099 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:41.426592 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a376b46b-87d4-45f7-8e7e-79a90889d800 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:41.426877 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a376b46b-87d4-45f7-8e7e-79a90889d800 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:41.427068 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a376b46b-87d4-45f7-8e7e-79a90889d800 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:41.428512 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 393/785] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:00:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 16:00:41.430246 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ee9acc-d33d-4450-8735-9e9e935ea87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:41.430921 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee9acc-d33d-4450-8735-9e9e935ea87e None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:41.431197 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee9acc-d33d-4450-8735-9e9e935ea87e None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:41.431467 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee9acc-d33d-4450-8735-9e9e935ea87e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:41.434770 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8618082b-9c02-4c9a-bd79-afdc72542e89 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:41.435289 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8618082b-9c02-4c9a-bd79-afdc72542e89 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:41.435762 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8618082b-9c02-4c9a-bd79-afdc72542e89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:41.445442 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ee9acc-d33d-4450-8735-9e9e935ea87e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:41.445442 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ee9acc-d33d-4450-8735-9e9e935ea87e None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:41.471263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-18ee9acc-d33d-4450-8735-9e9e935ea87e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:41.471263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:41.471263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:41.471263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:41.471263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:41.471263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:41.471263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:41.471263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:41.471263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:41.471263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:41.471263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:41.471263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:41.471263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:00:41.471263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:00:41.471263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:00:41.471263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:00:41.472251 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:41.472251 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:41.472251 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:41.472251 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:41.472251 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:41.472251 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:41.472251 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:41.472251 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:41.472251 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:41.472251 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:41.472251 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:41.472251 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:41.472251 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:41.472251 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:41.472251 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:41.472251 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:41.472251 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:00:41.473151 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:00:41.473151 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:41.473151 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:41.473151 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 393/786] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:00:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:41.473851 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-8618082b-9c02-4c9a-bd79-afdc72542e89 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:00:41.474788 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-8618082b-9c02-4c9a-bd79-afdc72542e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:00:41.477877 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ee9acc-d33d-4450-8735-9e9e935ea87e None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:41.611929 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 394/787] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:00:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 16:00:41.622365 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ee9acc-d33d-4450-8735-9e9e935ea87e None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:41.660028 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c830f07c-a470-4e1c-a5e4-8c5662f0701b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:41.660648 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c830f07c-a470-4e1c-a5e4-8c5662f0701b None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:41.660927 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c830f07c-a470-4e1c-a5e4-8c5662f0701b None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:41.661196 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c830f07c-a470-4e1c-a5e4-8c5662f0701b None admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:41.661842 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c830f07c-a470-4e1c-a5e4-8c5662f0701b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:41.662143 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c830f07c-a470-4e1c-a5e4-8c5662f0701b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:41.663564 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c830f07c-a470-4e1c-a5e4-8c5662f0701b None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:41.678508 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 394/788] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:00:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:00:41.683854 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8618082b-9c02-4c9a-bd79-afdc72542e89 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:41.822835 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8618082b-9c02-4c9a-bd79-afdc72542e89 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:41.833070 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:41.860842 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4612b1be-90a3-49b7-9c01-3e4be1c2d054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:41.861471 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4612b1be-90a3-49b7-9c01-3e4be1c2d054 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:41.861750 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4612b1be-90a3-49b7-9c01-3e4be1c2d054 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:41.862029 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4612b1be-90a3-49b7-9c01-3e4be1c2d054 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:41.869529 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4612b1be-90a3-49b7-9c01-3e4be1c2d054 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:41.870996 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4612b1be-90a3-49b7-9c01-3e4be1c2d054 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:41.881489 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8a85d8-bd92-4017-a30d-3c3df0305f53 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:41.881771 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8a85d8-bd92-4017-a30d-3c3df0305f53 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:41.882047 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8a85d8-bd92-4017-a30d-3c3df0305f53 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:41.883399 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 395/789] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:00:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 16:00:41.889726 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ca264-1a33-41f9-a345-b81a8d557c80 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:41.890010 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ca264-1a33-41f9-a345-b81a8d557c80 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:41.890286 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ca264-1a33-41f9-a345-b81a8d557c80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:41.893408 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-4612b1be-90a3-49b7-9c01-3e4be1c2d054 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 19 16:00:41.893408 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:41.893408 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:41.893408 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:41.893408 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:41.893408 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:41.893408 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:41.893408 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:41.893408 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:41.893408 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:41.893408 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:41.893408 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:41.893408 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 16:00:41.893408 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 16:00:41.893408 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 16:00:41.893408 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 16:00:41.894321 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:41.894321 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:41.894321 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 16:00:41.894321 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:00:41.894321 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 16:00:41.894321 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:00:41.894321 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:41.894321 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:41.894321 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:41.894321 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:41.894321 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:41.894321 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:41.894321 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:41.894321 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:41.894321 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:41.894321 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:41.894321 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:41.894321 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:41.895277 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:41.895277 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:41.895277 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 16:00:41.895277 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 16:00:41.895277 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 16:00:41.895277 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 16:00:41.895277 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 16:00:41.895277 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 16:00:41.895277 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 19 16:00:41.895277 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:41.897399 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 395/790] 10.222.0.57 () {56 vars in 1093 bytes} [Mon Sep 19 16:00:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:41.903359 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4612b1be-90a3-49b7-9c01-3e4be1c2d054 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:41.929496 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-cd4ca264-1a33-41f9-a345-b81a8d557c80 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:00:41.930971 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-cd4ca264-1a33-41f9-a345-b81a8d557c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:00:41.962264 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:42.024901 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 396/791] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:00:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 16:00:42.025784 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4ca264-1a33-41f9-a345-b81a8d557c80 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:42.052904 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4612b1be-90a3-49b7-9c01-3e4be1c2d054 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:42.126208 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f66451-0ba9-47e1-8133-460e3dc1f820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:42.127020 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f66451-0ba9-47e1-8133-460e3dc1f820 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:42.127302 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f66451-0ba9-47e1-8133-460e3dc1f820 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:42.127570 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f66451-0ba9-47e1-8133-460e3dc1f820 None admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:42.128319 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f66451-0ba9-47e1-8133-460e3dc1f820 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:42.128623 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f66451-0ba9-47e1-8133-460e3dc1f820 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:42.130084 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f66451-0ba9-47e1-8133-460e3dc1f820 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:42.164569 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 396/792] 10.222.0.57 () {56 vars in 1133 bytes} [Mon Sep 19 16:00:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:42.171390 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f66451-0ba9-47e1-8133-460e3dc1f820 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:42.177353 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4ca264-1a33-41f9-a345-b81a8d557c80 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:42.218866 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:42.233440 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb8226c-6e32-46dc-9ac2-6f67262f0aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:42.234066 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb8226c-6e32-46dc-9ac2-6f67262f0aa8 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:42.234340 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb8226c-6e32-46dc-9ac2-6f67262f0aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:42.234710 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb8226c-6e32-46dc-9ac2-6f67262f0aa8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:42.251701 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb8226c-6e32-46dc-9ac2-6f67262f0aa8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 16:00:42.260042 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-feb8226c-6e32-46dc-9ac2-6f67262f0aa8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 16:00:42.260042 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:42.260042 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:42.260042 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:42.260042 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:42.260042 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:42.260042 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:42.260042 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:42.260042 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:42.260042 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:42.260042 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:42.260042 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:42.260042 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 16:00:42.260042 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 16:00:42.260042 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 16:00:42.260042 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 16:00:42.260954 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 16:00:42.260954 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 16:00:42.260954 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 16:00:42.260954 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 16:00:42.260954 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:42.260954 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:42.260954 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:42.260954 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:42.260954 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:42.260954 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:42.260954 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:42.260954 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:42.260954 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:42.260954 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:42.260954 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:42.260954 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:42.261789 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:42.261789 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:42.261789 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:42.261789 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:42.261789 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 16:00:42.261789 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 16:00:42.261789 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 16:00:42.261789 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 16:00:42.261789 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 16:00:42.261789 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 16:00:42.261789 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 16:00:42.261789 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:42.263798 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 397/793] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Sep 19 16:00:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:42.267426 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb8226c-6e32-46dc-9ac2-6f67262f0aa8 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:42.294887 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:42.352740 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f66451-0ba9-47e1-8133-460e3dc1f820 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:42.389809 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ce05c3-eb37-4223-b2b9-cbf4b4bac45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:42.391074 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ce05c3-eb37-4223-b2b9-cbf4b4bac45f None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:42.391074 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ce05c3-eb37-4223-b2b9-cbf4b4bac45f None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:42.391074 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ce05c3-eb37-4223-b2b9-cbf4b4bac45f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:42.399640 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb8226c-6e32-46dc-9ac2-6f67262f0aa8 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:42.436521 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e4d785-5f92-42f8-87e1-63d1de409a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:42.437521 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e4d785-5f92-42f8-87e1-63d1de409a2f None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:42.438002 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e4d785-5f92-42f8-87e1-63d1de409a2f None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:42.438502 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e4d785-5f92-42f8-87e1-63d1de409a2f None admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:42.439415 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e4d785-5f92-42f8-87e1-63d1de409a2f None admin] RBAC: Adding query filter params (name=service) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:42.439955 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e4d785-5f92-42f8-87e1-63d1de409a2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:42.441590 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e4d785-5f92-42f8-87e1-63d1de409a2f None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:42.450994 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 398/794] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Sep 19 16:00:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:42.452254 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e4d785-5f92-42f8-87e1-63d1de409a2f None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:42.467430 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ce05c3-eb37-4223-b2b9-cbf4b4bac45f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:42.468818 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ce05c3-eb37-4223-b2b9-cbf4b4bac45f None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:42.495299 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:42.533282 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:42.565875 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e4d785-5f92-42f8-87e1-63d1de409a2f None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:42.581214 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 397/795] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 19 16:00:42 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 16:00:42.581954 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ce05c3-eb37-4223-b2b9-cbf4b4bac45f None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:42.613079 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d8c071-0e59-46f5-b9a0-c10b00b1360d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:42.613702 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d8c071-0e59-46f5-b9a0-c10b00b1360d None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:42.613980 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d8c071-0e59-46f5-b9a0-c10b00b1360d None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:42.614249 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d8c071-0e59-46f5-b9a0-c10b00b1360d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:42.628207 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d8c071-0e59-46f5-b9a0-c10b00b1360d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 16:00:42.633974 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-e7d8c071-0e59-46f5-b9a0-c10b00b1360d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 16:00:42.633974 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:42.633974 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:42.633974 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:42.633974 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:42.633974 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:42.633974 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:42.633974 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:42.633974 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:42.633974 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:42.633974 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:42.633974 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:42.633974 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 16:00:42.633974 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 16:00:42.633974 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 16:00:42.633974 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 16:00:42.634922 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 16:00:42.634922 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 16:00:42.634922 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 16:00:42.634922 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 16:00:42.634922 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:42.634922 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:42.634922 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:42.634922 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:42.634922 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:42.634922 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:42.634922 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:42.634922 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:42.634922 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:42.634922 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:42.634922 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:42.634922 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:42.635776 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:42.635776 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:42.635776 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:42.635776 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:42.635776 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 16:00:42.635776 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 16:00:42.635776 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 16:00:42.635776 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 16:00:42.635776 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 16:00:42.635776 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 16:00:42.635776 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 16:00:42.635776 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:42.638455 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 399/796] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 19 16:00:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:42.639509 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d8c071-0e59-46f5-b9a0-c10b00b1360d None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:42.727146 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ce05c3-eb37-4223-b2b9-cbf4b4bac45f None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:42.761866 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:42.774623 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d8c071-0e59-46f5-b9a0-c10b00b1360d None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:42.780741 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fafa012-ff3b-4354-ac07-4ee39bd02849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:42.781394 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fafa012-ff3b-4354-ac07-4ee39bd02849 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:42.781675 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fafa012-ff3b-4354-ac07-4ee39bd02849 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:42.782660 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fafa012-ff3b-4354-ac07-4ee39bd02849 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:42.795050 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fafa012-ff3b-4354-ac07-4ee39bd02849 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:42.796916 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fafa012-ff3b-4354-ac07-4ee39bd02849 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:42.813937 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-9fafa012-ff3b-4354-ac07-4ee39bd02849 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:42.813937 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:42.813937 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:42.813937 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:42.813937 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:42.813937 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:42.813937 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:42.813937 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:42.813937 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:42.813937 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:42.813937 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:42.813937 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:42.813937 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:00:42.813937 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:00:42.813937 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:00:42.813937 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:00:42.814973 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:42.814973 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:42.814973 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:42.814973 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:42.814973 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:42.814973 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:42.814973 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:42.814973 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:42.814973 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:42.814973 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:42.814973 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:42.814973 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:42.814973 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:42.814973 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:42.814973 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:42.814973 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:42.814973 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:00:42.815827 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:00:42.815827 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:42.815827 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:42.817829 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 398/797] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:00:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:42.818171 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac9d026-75bc-402a-98a3-b718e792778a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:42.819012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac9d026-75bc-402a-98a3-b718e792778a None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:42.819190 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fafa012-ff3b-4354-ac07-4ee39bd02849 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:42.819480 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac9d026-75bc-402a-98a3-b718e792778a None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:42.819947 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac9d026-75bc-402a-98a3-b718e792778a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:42.822418 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:42.835205 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac9d026-75bc-402a-98a3-b718e792778a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:42.837089 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac9d026-75bc-402a-98a3-b718e792778a None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:42.855251 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-8ac9d026-75bc-402a-98a3-b718e792778a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:42.855251 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:42.855251 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:42.855251 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:42.855251 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:42.855251 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:42.855251 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:42.855251 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:42.855251 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:42.855251 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:42.855251 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:42.855251 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:42.855251 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:00:42.855251 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:00:42.855251 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:00:42.855251 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:00:42.856815 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:42.856815 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:42.856815 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:42.856815 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:42.856815 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:42.856815 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:42.856815 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:42.856815 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:42.856815 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:42.856815 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:42.856815 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:42.856815 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:42.856815 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:42.856815 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:42.856815 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:42.856815 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:42.856815 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:00:42.857692 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:00:42.857692 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:42.857692 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:42.857692 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 400/798] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:00:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:42.858336 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac9d026-75bc-402a-98a3-b718e792778a None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:42.950691 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fafa012-ff3b-4354-ac07-4ee39bd02849 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:42.997466 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cf7a09-3a5a-4f6c-a0d8-faeb663cbc44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:42.998100 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf7a09-3a5a-4f6c-a0d8-faeb663cbc44 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:42.998398 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf7a09-3a5a-4f6c-a0d8-faeb663cbc44 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:42.998686 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf7a09-3a5a-4f6c-a0d8-faeb663cbc44 None admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:42.999366 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cf7a09-3a5a-4f6c-a0d8-faeb663cbc44 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:42.999671 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cf7a09-3a5a-4f6c-a0d8-faeb663cbc44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:43.002351 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cf7a09-3a5a-4f6c-a0d8-faeb663cbc44 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:43.015118 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac9d026-75bc-402a-98a3-b718e792778a None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:43.019936 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 399/799] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 16:00:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:43.019936 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cf7a09-3a5a-4f6c-a0d8-faeb663cbc44 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:43.071189 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1084c13d-1d91-4482-8549-37ab071db15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:43.071814 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1084c13d-1d91-4482-8549-37ab071db15f None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:43.072091 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1084c13d-1d91-4482-8549-37ab071db15f None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:43.072416 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1084c13d-1d91-4482-8549-37ab071db15f None admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:43.073068 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1084c13d-1d91-4482-8549-37ab071db15f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:43.073374 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1084c13d-1d91-4482-8549-37ab071db15f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:43.075327 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:43.076392 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1084c13d-1d91-4482-8549-37ab071db15f None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:43.094881 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 401/800] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:00:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:43.095976 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1084c13d-1d91-4482-8549-37ab071db15f None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:43.146417 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cf7a09-3a5a-4f6c-a0d8-faeb663cbc44 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:43.154097 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:43.182577 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc789d7-801c-4e4c-b94b-c20da4e342b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:43.183276 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc789d7-801c-4e4c-b94b-c20da4e342b0 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:43.183463 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc789d7-801c-4e4c-b94b-c20da4e342b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:43.183731 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc789d7-801c-4e4c-b94b-c20da4e342b0 None admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:43.184977 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc789d7-801c-4e4c-b94b-c20da4e342b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:43.185273 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc789d7-801c-4e4c-b94b-c20da4e342b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:43.186634 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc789d7-801c-4e4c-b94b-c20da4e342b0 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:43.198364 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 400/801] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:00:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:43.199236 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc789d7-801c-4e4c-b94b-c20da4e342b0 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:43.216089 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1084c13d-1d91-4482-8549-37ab071db15f None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:43.264041 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c68b13-7d2a-4573-9748-e1c92429393c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:43.264706 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c68b13-7d2a-4573-9748-e1c92429393c None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:43.264944 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c68b13-7d2a-4573-9748-e1c92429393c None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:43.265271 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c68b13-7d2a-4573-9748-e1c92429393c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:43.284194 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c68b13-7d2a-4573-9748-e1c92429393c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:43.287010 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c68b13-7d2a-4573-9748-e1c92429393c None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:43.317009 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-82c68b13-7d2a-4573-9748-e1c92429393c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:43.317009 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:43.317009 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:43.317009 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:43.317009 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:43.317009 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:43.317009 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:43.317009 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:43.317009 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:43.317009 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:43.317009 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:43.317009 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:43.317009 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:00:43.317009 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:00:43.317009 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:00:43.317009 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:00:43.318071 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:43.318071 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:43.318071 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:43.318071 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:43.318071 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:43.318071 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:43.318071 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:43.318071 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:43.318071 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:43.318071 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:43.318071 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:43.318071 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:43.318071 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:43.318071 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:43.318071 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:43.318071 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:43.318071 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:00:43.319021 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:00:43.319021 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:43.319021 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:43.319021 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 402/802] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:00:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:43.320905 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed3e576-f9ac-425e-bfa8-98d6839e060c None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:43.321356 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed3e576-f9ac-425e-bfa8-98d6839e060c None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:43.321832 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed3e576-f9ac-425e-bfa8-98d6839e060c None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:43.326422 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 403/803] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:00:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:00:43.326422 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed3e576-f9ac-425e-bfa8-98d6839e060c None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:43.329673 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:43.332972 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc789d7-801c-4e4c-b94b-c20da4e342b0 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:43.370667 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10614c86-40f5-4b0c-81b8-971e10af4dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:43.371593 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10614c86-40f5-4b0c-81b8-971e10af4dd8 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:43.371844 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10614c86-40f5-4b0c-81b8-971e10af4dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:43.372129 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10614c86-40f5-4b0c-81b8-971e10af4dd8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:43.380545 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10614c86-40f5-4b0c-81b8-971e10af4dd8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:43.382153 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10614c86-40f5-4b0c-81b8-971e10af4dd8 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:43.394318 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-10614c86-40f5-4b0c-81b8-971e10af4dd8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 16:00:43.394318 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:43.394318 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:43.394318 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:43.394318 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:43.394318 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:43.394318 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:43.394318 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:43.394318 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:43.394318 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:43.394318 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:43.394318 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:43.394318 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 16:00:43.394318 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 16:00:43.394318 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 16:00:43.394318 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 16:00:43.399321 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:43.399321 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:43.399321 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 16:00:43.399321 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:00:43.399321 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 16:00:43.399321 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:00:43.399321 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:43.399321 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:43.399321 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:43.399321 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:43.399321 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:43.399321 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:43.399321 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:43.399321 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:43.399321 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:43.399321 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:43.399321 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:43.399321 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:43.400636 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:43.400636 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:43.400636 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 16:00:43.400636 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 16:00:43.400636 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 16:00:43.400636 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 16:00:43.400636 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 16:00:43.400636 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 16:00:43.400636 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 16:00:43.400636 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:43.400636 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 401/804] 10.222.0.57 () {56 vars in 1117 bytes} [Mon Sep 19 16:00:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:43.400636 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10614c86-40f5-4b0c-81b8-971e10af4dd8 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:43.455433 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:43.460582 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed3e576-f9ac-425e-bfa8-98d6839e060c None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:43.497928 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e79a60-3c1a-4766-bb3c-d3d162a4c34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:43.498619 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e79a60-3c1a-4766-bb3c-d3d162a4c34a None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:43.498899 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e79a60-3c1a-4766-bb3c-d3d162a4c34a None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:43.499184 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e79a60-3c1a-4766-bb3c-d3d162a4c34a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:43.504373 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10614c86-40f5-4b0c-81b8-971e10af4dd8 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:43.506548 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e79a60-3c1a-4766-bb3c-d3d162a4c34a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:43.508041 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e79a60-3c1a-4766-bb3c-d3d162a4c34a None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:43.520274 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-e0e79a60-3c1a-4766-bb3c-d3d162a4c34a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:00:43.520274 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:43.520274 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:43.520274 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:43.520274 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:43.520274 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:43.520274 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:43.520274 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:43.520274 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:43.520274 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:43.520274 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:43.520274 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:43.520274 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 16:00:43.520274 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 16:00:43.520274 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 16:00:43.521238 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 16:00:43.521238 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:43.521238 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:43.521238 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:43.521238 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:43.521238 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:43.521238 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:43.521238 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:43.521238 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:43.521238 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:43.521238 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:43.521238 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:43.521238 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:43.521238 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:43.521238 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:43.521238 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:43.521238 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:43.522116 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 16:00:43.522116 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 16:00:43.522116 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 16:00:43.522116 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 16:00:43.522116 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 16:00:43.522116 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 16:00:43.522116 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:00:43.522116 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:43.522116 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 404/805] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 19 16:00:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:43.522624 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5544febc-6ffc-45cb-9009-47539f4fdbce None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:43.522854 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5544febc-6ffc-45cb-9009-47539f4fdbce None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:43.523171 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5544febc-6ffc-45cb-9009-47539f4fdbce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:43.542747 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca71cfd9-5f9b-428d-8dd2-694fcab7e4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:43.543391 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-5544febc-6ffc-45cb-9009-47539f4fdbce None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:00:43.544066 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71cfd9-5f9b-428d-8dd2-694fcab7e4cf None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:43.544351 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71cfd9-5f9b-428d-8dd2-694fcab7e4cf None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:43.544626 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71cfd9-5f9b-428d-8dd2-694fcab7e4cf None admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:43.545141 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-5544febc-6ffc-45cb-9009-47539f4fdbce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:00:43.545284 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca71cfd9-5f9b-428d-8dd2-694fcab7e4cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:43.545593 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca71cfd9-5f9b-428d-8dd2-694fcab7e4cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:43.547042 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca71cfd9-5f9b-428d-8dd2-694fcab7e4cf None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:43.559846 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 402/806] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:00:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:43.561512 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dccb67-50f7-48f8-ab37-e877fad3f3d2 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:43.561799 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dccb67-50f7-48f8-ab37-e877fad3f3d2 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:43.562080 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dccb67-50f7-48f8-ab37-e877fad3f3d2 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:43.563414 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 403/807] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:00:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:00:43.564129 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dccb67-50f7-48f8-ab37-e877fad3f3d2 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:43.588750 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:43.626040 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 405/808] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:00:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 16:00:43.626924 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5544febc-6ffc-45cb-9009-47539f4fdbce None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:43.688467 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dccb67-50f7-48f8-ab37-e877fad3f3d2 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:43.728034 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abc1ab1-b45b-4694-93ee-1415080ec0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:43.728643 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc1ab1-b45b-4694-93ee-1415080ec0cd None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:43.728922 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc1ab1-b45b-4694-93ee-1415080ec0cd None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:43.729194 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc1ab1-b45b-4694-93ee-1415080ec0cd None admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:43.729926 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abc1ab1-b45b-4694-93ee-1415080ec0cd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:43.730059 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:43.730244 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abc1ab1-b45b-4694-93ee-1415080ec0cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:43.732837 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abc1ab1-b45b-4694-93ee-1415080ec0cd None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:43.744694 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5544febc-6ffc-45cb-9009-47539f4fdbce None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:43.758231 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 404/809] 10.222.0.57 () {56 vars in 1149 bytes} [Mon Sep 19 16:00:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:43.758231 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abc1ab1-b45b-4694-93ee-1415080ec0cd None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:43.788450 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15d672f-1e9b-40be-b5eb-101e28e9151c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:43.789062 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d672f-1e9b-40be-b5eb-101e28e9151c None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:43.789341 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d672f-1e9b-40be-b5eb-101e28e9151c None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:43.789610 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d672f-1e9b-40be-b5eb-101e28e9151c None admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:43.790294 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15d672f-1e9b-40be-b5eb-101e28e9151c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:43.790623 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15d672f-1e9b-40be-b5eb-101e28e9151c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:43.792296 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15d672f-1e9b-40be-b5eb-101e28e9151c None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:43.804337 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 406/810] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 19 16:00:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:43.805592 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469985f-a7ca-4c0a-bb58-985b72810470 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:43.805875 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469985f-a7ca-4c0a-bb58-985b72810470 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:43.806149 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469985f-a7ca-4c0a-bb58-985b72810470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:43.824321 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-6469985f-a7ca-4c0a-bb58-985b72810470 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:00:43.825614 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-6469985f-a7ca-4c0a-bb58-985b72810470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:00:43.852544 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:43.877345 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abc1ab1-b45b-4694-93ee-1415080ec0cd None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:43.894938 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 407/811] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:00:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 16:00:43.895812 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6469985f-a7ca-4c0a-bb58-985b72810470 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:43.911818 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caad455-ca43-490f-9dd5-5164255c9279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:43.912449 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caad455-ca43-490f-9dd5-5164255c9279 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:43.912728 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caad455-ca43-490f-9dd5-5164255c9279 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:43.913003 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caad455-ca43-490f-9dd5-5164255c9279 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:43.922597 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caad455-ca43-490f-9dd5-5164255c9279 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:43.924390 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caad455-ca43-490f-9dd5-5164255c9279 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:43.937556 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-2caad455-ca43-490f-9dd5-5164255c9279 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 19 16:00:43.937556 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:43.937556 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:43.937556 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:43.937556 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:43.937556 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:43.937556 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:43.937556 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:43.937556 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:43.937556 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:43.937556 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:43.937556 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:43.937556 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 16:00:43.937556 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 16:00:43.937556 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 16:00:43.937556 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 16:00:43.938624 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:43.938624 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:43.938624 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 16:00:43.938624 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:00:43.938624 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 16:00:43.938624 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:00:43.938624 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:43.938624 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:43.938624 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:43.938624 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:43.938624 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:43.938624 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:43.938624 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:43.938624 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:43.938624 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:43.938624 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:43.938624 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:43.938624 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:43.939551 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:43.939551 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:43.939551 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 16:00:43.939551 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 16:00:43.939551 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 16:00:43.939551 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 16:00:43.939551 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 16:00:43.939551 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 16:00:43.939551 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 19 16:00:43.939551 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:43.939551 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 405/812] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 19 16:00:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:43.943936 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caad455-ca43-490f-9dd5-5164255c9279 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:43.994733 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:44.018884 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6469985f-a7ca-4c0a-bb58-985b72810470 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:44.049117 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caad455-ca43-490f-9dd5-5164255c9279 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:44.057598 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06252a5f-e761-4dce-a6c3-c69718b52d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:44.058238 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06252a5f-e761-4dce-a6c3-c69718b52d25 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:44.058638 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06252a5f-e761-4dce-a6c3-c69718b52d25 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:44.058954 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06252a5f-e761-4dce-a6c3-c69718b52d25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:44.070354 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06252a5f-e761-4dce-a6c3-c69718b52d25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:44.071820 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06252a5f-e761-4dce-a6c3-c69718b52d25 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:44.084662 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a88b59-9590-4b2d-b80d-8f393ac3f7d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:44.085353 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a88b59-9590-4b2d-b80d-8f393ac3f7d1 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:44.085637 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a88b59-9590-4b2d-b80d-8f393ac3f7d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:44.085915 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a88b59-9590-4b2d-b80d-8f393ac3f7d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:44.086729 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a88b59-9590-4b2d-b80d-8f393ac3f7d1 None admin] RBAC: Adding query filter params (role.id=63d2d197bfd6497583e94bfb065ce29b, scope.project.id=be03e5300b3343169f35c643c93d1e06, user.id=1ecd01cc72834b72b5ff2c4b47249737) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:44.087045 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a88b59-9590-4b2d-b80d-8f393ac3f7d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:44.087459 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-06252a5f-e761-4dce-a6c3-c69718b52d25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:44.087459 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:44.087459 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:44.087459 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:44.087459 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:44.087459 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:44.087459 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:44.087459 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:44.087459 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:44.087459 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:44.087459 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:44.087459 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:44.087459 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:00:44.087459 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:00:44.087459 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:00:44.087459 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:00:44.088361 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:44.088361 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:44.088361 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:44.088361 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:44.088361 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:44.088361 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:44.088361 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:44.088361 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:44.088361 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:44.088361 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:44.088361 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:44.088361 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:44.088361 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:44.088361 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:44.088361 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:44.088361 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:44.088361 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:00:44.089232 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:00:44.089232 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:44.089232 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:44.089409 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a88b59-9590-4b2d-b80d-8f393ac3f7d1 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:44.092590 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:44.093630 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 408/813] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:00:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:44.094660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06252a5f-e761-4dce-a6c3-c69718b52d25 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:44.102577 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 406/814] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 19 16:00:43 2022] GET /identity/v3/role_assignments?user.id=1ecd01cc72834b72b5ff2c4b47249737&scope.project.id=be03e5300b3343169f35c643c93d1e06&role.id=63d2d197bfd6497583e94bfb065ce29b => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:44.103994 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a88b59-9590-4b2d-b80d-8f393ac3f7d1 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:44.211885 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06252a5f-e761-4dce-a6c3-c69718b52d25 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:44.227288 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a88b59-9590-4b2d-b80d-8f393ac3f7d1 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:44.248974 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:44.257920 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e66a8-5648-401e-a771-1a4cb5c18e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:44.258627 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e66a8-5648-401e-a771-1a4cb5c18e6c None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:44.258910 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e66a8-5648-401e-a771-1a4cb5c18e6c None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:44.259500 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e66a8-5648-401e-a771-1a4cb5c18e6c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:44.260587 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0e66a8-5648-401e-a771-1a4cb5c18e6c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:44.263586 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0e66a8-5648-401e-a771-1a4cb5c18e6c None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:44.268673 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2851163-9a51-4f12-8bcc-c0b3d18afb9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:44.269296 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2851163-9a51-4f12-8bcc-c0b3d18afb9b None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:44.269578 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2851163-9a51-4f12-8bcc-c0b3d18afb9b None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:44.269852 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2851163-9a51-4f12-8bcc-c0b3d18afb9b None admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:44.270644 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2851163-9a51-4f12-8bcc-c0b3d18afb9b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:44.270936 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2851163-9a51-4f12-8bcc-c0b3d18afb9b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:44.272391 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2851163-9a51-4f12-8bcc-c0b3d18afb9b None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:44.273508 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-ae0e66a8-5648-401e-a771-1a4cb5c18e6c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 19 16:00:44.273508 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:44.273508 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 16:00:44.273508 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 16:00:44.273508 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 16:00:44.273508 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 16:00:44.273508 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 16:00:44.273508 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 16:00:44.273508 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 19 16:00:44.273508 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:44.273508 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 16:00:44.273508 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:44.273508 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:44.273508 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:44.273508 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:44.273508 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:44.273508 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:44.274819 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:44.274819 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:44.274819 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:44.274819 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:44.274819 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:44.274819 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:44.274819 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 16:00:44.274819 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 16:00:44.274819 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 16:00:44.274819 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 16:00:44.274819 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:44.274819 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:44.274819 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:44.274819 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:44.274819 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:44.274819 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:44.275630 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:44.275630 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:44.275630 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:44.275630 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:44.275630 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:44.275630 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:44.275630 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:44.275630 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:44.275630 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:44.275630 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:44.275630 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 16:00:44.275630 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 16:00:44.275630 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 19 16:00:44.275630 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:44.275630 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 409/815] 10.222.0.57 () {56 vars in 1129 bytes} [Mon Sep 19 16:00:44 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:44.276407 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e66a8-5648-401e-a771-1a4cb5c18e6c None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:44.297926 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 407/816] 10.222.0.57 () {56 vars in 1135 bytes} [Mon Sep 19 16:00:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:44.299266 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2851163-9a51-4f12-8bcc-c0b3d18afb9b None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:44.366448 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:44.405645 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e66a8-5648-401e-a771-1a4cb5c18e6c None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:44.412890 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2851163-9a51-4f12-8bcc-c0b3d18afb9b None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:44.444964 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e5402b-2665-4eb5-89a0-1036c5a2a9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:44.445575 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e5402b-2665-4eb5-89a0-1036c5a2a9d5 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:44.445855 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e5402b-2665-4eb5-89a0-1036c5a2a9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:44.446128 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e5402b-2665-4eb5-89a0-1036c5a2a9d5 None admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:44.446848 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e5402b-2665-4eb5-89a0-1036c5a2a9d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:44.447153 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e5402b-2665-4eb5-89a0-1036c5a2a9d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:44.448800 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e5402b-2665-4eb5-89a0-1036c5a2a9d5 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:44.450554 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4fa62d-5ea5-47fd-8f79-d2dfc90d6772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:44.451184 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4fa62d-5ea5-47fd-8f79-d2dfc90d6772 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:44.451467 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4fa62d-5ea5-47fd-8f79-d2dfc90d6772 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:44.451736 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4fa62d-5ea5-47fd-8f79-d2dfc90d6772 None admin] PATH_INFO: `/v3/services/network` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:44.452306 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4fa62d-5ea5-47fd-8f79-d2dfc90d6772 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:44.453648 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4fa62d-5ea5-47fd-8f79-d2dfc90d6772 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:44.466570 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-6e4fa62d-5ea5-47fd-8f79-d2dfc90d6772 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 19 16:00:44.466570 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:44.466570 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 16:00:44.466570 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 16:00:44.466570 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 16:00:44.466570 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 16:00:44.466570 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 16:00:44.466570 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 16:00:44.466570 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 19 16:00:44.466570 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:44.466570 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 16:00:44.466570 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:44.466570 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:44.466570 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:44.466570 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:44.466570 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:44.466570 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:44.467598 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:44.467598 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:44.467598 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:44.467598 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:44.467598 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:44.467598 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:44.467598 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 16:00:44.467598 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 16:00:44.467598 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 16:00:44.467598 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 16:00:44.467598 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:44.467598 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:44.467598 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:44.467598 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:44.467598 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:44.467598 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:44.468419 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:44.468419 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:44.468419 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:44.468419 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:44.468419 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:44.468419 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:44.468419 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:44.468419 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:44.468419 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:44.468419 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:44.468419 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 16:00:44.468419 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 16:00:44.468419 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 19 16:00:44.468419 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:44.468419 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 408/817] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 19 16:00:44 2022] GET /identity/v3/services/network => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:44.470348 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4fa62d-5ea5-47fd-8f79-d2dfc90d6772 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:44.470451 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 410/818] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:00:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:44.470451 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e5402b-2665-4eb5-89a0-1036c5a2a9d5 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:44.530714 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:44.583105 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4fa62d-5ea5-47fd-8f79-d2dfc90d6772 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:44.583404 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e5402b-2665-4eb5-89a0-1036c5a2a9d5 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:44.620125 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:44.620858 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c213d3-90af-45d4-878c-ed1198841ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:44.622973 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c213d3-90af-45d4-878c-ed1198841ec6 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:44.623285 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c213d3-90af-45d4-878c-ed1198841ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:44.623583 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c213d3-90af-45d4-878c-ed1198841ec6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:44.635607 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5931ab10-3fc2-4944-8eac-7018ec3d9925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:44.636510 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5931ab10-3fc2-4944-8eac-7018ec3d9925 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:44.636510 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5931ab10-3fc2-4944-8eac-7018ec3d9925 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:44.636764 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5931ab10-3fc2-4944-8eac-7018ec3d9925 None admin] PATH_INFO: `/v3/services` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:44.637390 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5931ab10-3fc2-4944-8eac-7018ec3d9925 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:44.637697 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5931ab10-3fc2-4944-8eac-7018ec3d9925 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:44.639277 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5931ab10-3fc2-4944-8eac-7018ec3d9925 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:44.640299 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c213d3-90af-45d4-878c-ed1198841ec6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:44.641768 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c213d3-90af-45d4-878c-ed1198841ec6 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:44.650023 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 409/819] 10.222.0.57 () {56 vars in 1124 bytes} [Mon Sep 19 16:00:44 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:44.650023 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5931ab10-3fc2-4944-8eac-7018ec3d9925 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:44.659742 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-36c213d3-90af-45d4-878c-ed1198841ec6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:44.659742 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:44.659742 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:44.659742 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:44.659742 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:44.659742 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:44.659742 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:44.659742 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:44.659742 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:44.659742 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:44.659742 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:44.659742 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:44.659742 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:00:44.659742 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:00:44.659742 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:00:44.659742 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:00:44.660845 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:44.660845 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:44.660845 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:44.660845 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:44.660845 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:44.660845 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:44.660845 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:44.660845 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:44.660845 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:44.660845 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:44.660845 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:44.660845 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:44.660845 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:44.660845 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:44.660845 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:44.660845 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:44.660845 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:00:44.661728 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:00:44.661728 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:44.661728 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:44.661728 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 411/820] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:00:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:44.662461 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c213d3-90af-45d4-878c-ed1198841ec6 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:44.766098 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5931ab10-3fc2-4944-8eac-7018ec3d9925 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:44.779975 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c213d3-90af-45d4-878c-ed1198841ec6 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:44.787683 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:44.803107 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef95c7b-b5b1-475f-9150-14dca5818a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:44.803713 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef95c7b-b5b1-475f-9150-14dca5818a29 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:44.803990 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef95c7b-b5b1-475f-9150-14dca5818a29 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:44.804264 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef95c7b-b5b1-475f-9150-14dca5818a29 None admin] PATH_INFO: `/v3/services` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:44.804891 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef95c7b-b5b1-475f-9150-14dca5818a29 None admin] RBAC: Adding query filter params (name=network) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:44.805194 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef95c7b-b5b1-475f-9150-14dca5818a29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:44.806584 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef95c7b-b5b1-475f-9150-14dca5818a29 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:44.811800 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 410/821] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Sep 19 16:00:44 2022] GET /identity/v3/services?name=network => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:44.812819 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef95c7b-b5b1-475f-9150-14dca5818a29 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:44.823262 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17aa356-08fe-438f-bac9-a5e1ffebdeb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:44.824269 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17aa356-08fe-438f-bac9-a5e1ffebdeb4 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:44.824593 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17aa356-08fe-438f-bac9-a5e1ffebdeb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:44.824873 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17aa356-08fe-438f-bac9-a5e1ffebdeb4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:44.832860 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17aa356-08fe-438f-bac9-a5e1ffebdeb4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:44.834338 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17aa356-08fe-438f-bac9-a5e1ffebdeb4 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:44.847411 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-b17aa356-08fe-438f-bac9-a5e1ffebdeb4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:00:44.847411 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:44.847411 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:44.847411 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:44.847411 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:44.847411 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:44.847411 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:44.847411 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:44.847411 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:44.847411 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:44.847411 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:44.847411 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:44.847411 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 16:00:44.847411 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 16:00:44.847411 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 16:00:44.848477 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 16:00:44.848477 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:44.848477 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:44.848477 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:44.848477 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:44.848477 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:44.848477 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:44.848477 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:44.848477 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:44.848477 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:44.848477 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:44.848477 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:44.848477 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:44.848477 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:44.848477 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:44.848477 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:44.848477 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:44.849579 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 16:00:44.849579 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 16:00:44.849579 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 16:00:44.849579 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 16:00:44.849579 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 16:00:44.849579 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 16:00:44.849579 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:00:44.849579 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:44.851436 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 412/822] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 19 16:00:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:44.852424 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17aa356-08fe-438f-bac9-a5e1ffebdeb4 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:44.869960 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:44.926532 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef95c7b-b5b1-475f-9150-14dca5818a29 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:44.961109 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747420c2-5883-4c39-9d28-38385d85d28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:44.961750 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747420c2-5883-4c39-9d28-38385d85d28f None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:44.962051 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747420c2-5883-4c39-9d28-38385d85d28f None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:44.962306 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747420c2-5883-4c39-9d28-38385d85d28f None admin] PATH_INFO: `/v3/services` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:44.962970 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747420c2-5883-4c39-9d28-38385d85d28f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:44.963277 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747420c2-5883-4c39-9d28-38385d85d28f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:44.964731 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747420c2-5883-4c39-9d28-38385d85d28f None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:44.973425 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 411/823] 10.222.0.57 () {56 vars in 1124 bytes} [Mon Sep 19 16:00:44 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:44.974504 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747420c2-5883-4c39-9d28-38385d85d28f None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:44.986099 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17aa356-08fe-438f-bac9-a5e1ffebdeb4 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:45.036744 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eed18ef-862a-4560-aba1-4f2aea46ff8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:45.037830 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eed18ef-862a-4560-aba1-4f2aea46ff8d None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:45.038117 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eed18ef-862a-4560-aba1-4f2aea46ff8d None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:45.038429 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eed18ef-862a-4560-aba1-4f2aea46ff8d None admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:45.039093 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eed18ef-862a-4560-aba1-4f2aea46ff8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:45.039399 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eed18ef-862a-4560-aba1-4f2aea46ff8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:45.041158 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eed18ef-862a-4560-aba1-4f2aea46ff8d None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:45.057908 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 413/824] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:00:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:45.058821 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eed18ef-862a-4560-aba1-4f2aea46ff8d None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:45.089816 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:45.115250 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747420c2-5883-4c39-9d28-38385d85d28f None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:45.128786 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:45.149420 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb7fad2-0830-41c1-8d02-4bd06e620029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:45.150029 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb7fad2-0830-41c1-8d02-4bd06e620029 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:45.150312 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb7fad2-0830-41c1-8d02-4bd06e620029 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:45.150613 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb7fad2-0830-41c1-8d02-4bd06e620029 None admin] PATH_INFO: `/v3/services` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:45.151244 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb7fad2-0830-41c1-8d02-4bd06e620029 None admin] RBAC: Adding query filter params (type=network) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:45.151551 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb7fad2-0830-41c1-8d02-4bd06e620029 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:45.153437 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb7fad2-0830-41c1-8d02-4bd06e620029 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:45.162644 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 412/825] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Sep 19 16:00:44 2022] GET /identity/v3/services?type=network => generated 350 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:45.162644 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb7fad2-0830-41c1-8d02-4bd06e620029 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:45.174610 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eed18ef-862a-4560-aba1-4f2aea46ff8d None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:45.210021 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158b14f0-53f3-4753-97da-ef7f6d71bbac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:45.210658 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b14f0-53f3-4753-97da-ef7f6d71bbac None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:45.210938 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b14f0-53f3-4753-97da-ef7f6d71bbac None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:45.211210 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b14f0-53f3-4753-97da-ef7f6d71bbac None admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:45.211896 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158b14f0-53f3-4753-97da-ef7f6d71bbac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:45.212202 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158b14f0-53f3-4753-97da-ef7f6d71bbac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:45.213619 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158b14f0-53f3-4753-97da-ef7f6d71bbac None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:45.226544 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 414/826] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 19 16:00:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:45.227638 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158b14f0-53f3-4753-97da-ef7f6d71bbac None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:45.269805 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb7fad2-0830-41c1-8d02-4bd06e620029 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:45.305370 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91826b84-f7fa-4092-8230-f9632b2e3b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:45.306123 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91826b84-f7fa-4092-8230-f9632b2e3b82 None admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:45.306534 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91826b84-f7fa-4092-8230-f9632b2e3b82 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:45.306809 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91826b84-f7fa-4092-8230-f9632b2e3b82 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:45.307478 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91826b84-f7fa-4092-8230-f9632b2e3b82 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:45.308904 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91826b84-f7fa-4092-8230-f9632b2e3b82 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:45.324204 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:45.338600 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 413/827] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:00:45 2022] POST /identity/v3/endpoints => generated 346 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:00:45.343359 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158b14f0-53f3-4753-97da-ef7f6d71bbac None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:45.351433 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91826b84-f7fa-4092-8230-f9632b2e3b82 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:45.398771 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:45.407227 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b92a9b-03ab-46f2-8bf7-979a85552812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:45.408219 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b92a9b-03ab-46f2-8bf7-979a85552812 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:45.408656 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b92a9b-03ab-46f2-8bf7-979a85552812 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:45.409073 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b92a9b-03ab-46f2-8bf7-979a85552812 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:45.417221 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b92a9b-03ab-46f2-8bf7-979a85552812 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:45.419113 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b92a9b-03ab-46f2-8bf7-979a85552812 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:45.432228 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-e6b92a9b-03ab-46f2-8bf7-979a85552812 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:00:45.432228 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:45.432228 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:45.432228 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:45.432228 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:45.432228 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:45.432228 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:45.432228 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:45.432228 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:45.432228 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:45.432228 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:45.432228 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:45.432228 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 16:00:45.432228 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 16:00:45.432228 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 16:00:45.433280 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 16:00:45.433280 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:45.433280 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:45.433280 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:45.433280 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:45.433280 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:45.433280 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:45.433280 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:45.433280 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:45.433280 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:45.433280 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:45.433280 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:45.433280 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:45.433280 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:45.433280 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:45.433280 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:45.433280 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:45.434470 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 16:00:45.434470 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 16:00:45.434470 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 16:00:45.434470 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 16:00:45.434470 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 16:00:45.434470 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 16:00:45.434470 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:00:45.434470 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:45.436759 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 415/828] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 19 16:00:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:45.437782 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b92a9b-03ab-46f2-8bf7-979a85552812 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:45.503715 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91826b84-f7fa-4092-8230-f9632b2e3b82 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:45.552847 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd7eb3d-119a-4e5e-ba9d-5bb073475656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:45.553800 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd7eb3d-119a-4e5e-ba9d-5bb073475656 None admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:45.554241 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd7eb3d-119a-4e5e-ba9d-5bb073475656 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:45.554568 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd7eb3d-119a-4e5e-ba9d-5bb073475656 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:45.555518 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd7eb3d-119a-4e5e-ba9d-5bb073475656 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:45.557222 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd7eb3d-119a-4e5e-ba9d-5bb073475656 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:45.575190 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b92a9b-03ab-46f2-8bf7-979a85552812 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:45.588796 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 414/829] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:00:45 2022] POST /identity/v3/endpoints => generated 337 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:00:45.589746 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd7eb3d-119a-4e5e-ba9d-5bb073475656 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:45.635279 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b132764-0f05-4a5a-bed7-7156b16a0d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:45.636077 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b132764-0f05-4a5a-bed7-7156b16a0d27 None admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:45.636477 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b132764-0f05-4a5a-bed7-7156b16a0d27 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:45.636975 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b132764-0f05-4a5a-bed7-7156b16a0d27 None admin] PATH_INFO: `/v3/projects/be03e5300b3343169f35c643c93d1e06/users/7222bd3b313b47e5a4fa0ebc9ff0cdd5/roles/1e475e6a170a4dbabc4ce8348eeb30b9` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:45.653573 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:45.664699 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b132764-0f05-4a5a-bed7-7156b16a0d27 None admin] RBAC: Authorizing `identity:create_grant(project_id=be03e5300b3343169f35c643c93d1e06, user_id=7222bd3b313b47e5a4fa0ebc9ff0cdd5, role_id=1e475e6a170a4dbabc4ce8348eeb30b9)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:45.667292 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b132764-0f05-4a5a-bed7-7156b16a0d27 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:45.688160 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 416/830] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Sep 19 16:00:45 2022] PUT /identity/v3/projects/be03e5300b3343169f35c643c93d1e06/users/7222bd3b313b47e5a4fa0ebc9ff0cdd5/roles/1e475e6a170a4dbabc4ce8348eeb30b9 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:00:45.717529 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:45.734001 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd7eb3d-119a-4e5e-ba9d-5bb073475656 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:45.793258 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5562ab-8a73-4e35-84e8-fdd097b71724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:45.793908 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5562ab-8a73-4e35-84e8-fdd097b71724 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:45.794244 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5562ab-8a73-4e35-84e8-fdd097b71724 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:45.794496 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5562ab-8a73-4e35-84e8-fdd097b71724 None admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:45.795313 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5562ab-8a73-4e35-84e8-fdd097b71724 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:45.795699 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5562ab-8a73-4e35-84e8-fdd097b71724 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:45.798168 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5562ab-8a73-4e35-84e8-fdd097b71724 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:45.819753 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 415/831] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 19 16:00:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:45.825804 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b132764-0f05-4a5a-bed7-7156b16a0d27 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:45.964570 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b132764-0f05-4a5a-bed7-7156b16a0d27 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:46.007445 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca9237a-3408-4cff-a9b6-349b81d2f338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:46.008540 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca9237a-3408-4cff-a9b6-349b81d2f338 None admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:46.009056 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca9237a-3408-4cff-a9b6-349b81d2f338 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:46.009547 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca9237a-3408-4cff-a9b6-349b81d2f338 None admin] PATH_INFO: `/v3/projects/be03e5300b3343169f35c643c93d1e06/users/884a0524aca744b99c7496c11c3291ae/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:46.015060 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca9237a-3408-4cff-a9b6-349b81d2f338 None admin] RBAC: Authorizing `identity:create_grant(project_id=be03e5300b3343169f35c643c93d1e06, user_id=884a0524aca744b99c7496c11c3291ae, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:46.016555 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca9237a-3408-4cff-a9b6-349b81d2f338 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:46.037236 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 417/832] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Sep 19 16:00:45 2022] PUT /identity/v3/projects/be03e5300b3343169f35c643c93d1e06/users/884a0524aca744b99c7496c11c3291ae/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:00:46.444715 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e1ccf9-cc3f-4c48-8eef-240a88bcc8f6 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:46.445215 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e1ccf9-cc3f-4c48-8eef-240a88bcc8f6 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:46.445215 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e1ccf9-cc3f-4c48-8eef-240a88bcc8f6 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:46.446807 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 416/833] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:00:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:00:46.452933 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ca6f64-bfb0-49c1-96e7-6b95eb72e533 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:46.453369 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ca6f64-bfb0-49c1-96e7-6b95eb72e533 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:46.453550 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ca6f64-bfb0-49c1-96e7-6b95eb72e533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:46.472533 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-a6ca6f64-bfb0-49c1-96e7-6b95eb72e533 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:00:46.473781 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-a6ca6f64-bfb0-49c1-96e7-6b95eb72e533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:00:46.522126 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:46.551877 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 418/834] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:00:46 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 16:00:46.833821 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e1ccf9-cc3f-4c48-8eef-240a88bcc8f6 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:46.952008 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e1ccf9-cc3f-4c48-8eef-240a88bcc8f6 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:46.991024 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4fca5b-d04b-4bc1-ab09-decae57e1d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:46.991658 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4fca5b-d04b-4bc1-ab09-decae57e1d0d None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:46.991937 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4fca5b-d04b-4bc1-ab09-decae57e1d0d None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:46.992207 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4fca5b-d04b-4bc1-ab09-decae57e1d0d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:46.992795 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4fca5b-d04b-4bc1-ab09-decae57e1d0d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:46.994165 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4fca5b-d04b-4bc1-ab09-decae57e1d0d None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:47.004021 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-0f4fca5b-d04b-4bc1-ab09-decae57e1d0d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 19 16:00:47.004021 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:47.004021 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 16:00:47.004021 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 16:00:47.004021 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 16:00:47.004021 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 16:00:47.004021 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 16:00:47.004021 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 16:00:47.004021 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 19 16:00:47.004021 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:47.004021 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 16:00:47.004021 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:47.004021 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:47.004021 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:47.004021 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:47.004021 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:47.005048 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:47.005048 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:47.005048 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:47.005048 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:47.005048 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:47.005048 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:47.005048 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:47.005048 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 16:00:47.005048 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 16:00:47.005048 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 16:00:47.005048 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 16:00:47.005048 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:47.005048 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:47.005048 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:47.005048 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:47.005048 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:47.005048 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:47.005993 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:47.005993 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:47.005993 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:47.005993 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:47.005993 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:47.005993 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:47.005993 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:47.005993 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:47.005993 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:47.005993 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:47.005993 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 16:00:47.005993 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 16:00:47.005993 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 19 16:00:47.005993 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:47.005993 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 417/835] 10.222.0.57 () {56 vars in 1132 bytes} [Mon Sep 19 16:00:46 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 16:00:47.011218 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ca6f64-bfb0-49c1-96e7-6b95eb72e533 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:47.115950 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ca6f64-bfb0-49c1-96e7-6b95eb72e533 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:47.153790 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99de4dca-5497-4efd-a377-fd054922ce8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:47.154420 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99de4dca-5497-4efd-a377-fd054922ce8c None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:47.154706 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99de4dca-5497-4efd-a377-fd054922ce8c None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:47.154980 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99de4dca-5497-4efd-a377-fd054922ce8c None admin] PATH_INFO: `/v3/services` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:47.155615 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99de4dca-5497-4efd-a377-fd054922ce8c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:47.155920 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99de4dca-5497-4efd-a377-fd054922ce8c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:47.157285 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99de4dca-5497-4efd-a377-fd054922ce8c None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:47.166734 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 419/836] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Sep 19 16:00:47 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:47.170004 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4fca5b-d04b-4bc1-ab09-decae57e1d0d None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:47.186673 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:47.275392 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4fca5b-d04b-4bc1-ab09-decae57e1d0d None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:47.311588 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842b4636-1564-49d6-802f-ce4cafa324b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:47.312511 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b4636-1564-49d6-802f-ce4cafa324b6 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:47.312996 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b4636-1564-49d6-802f-ce4cafa324b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:47.313458 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b4636-1564-49d6-802f-ce4cafa324b6 None admin] PATH_INFO: `/v3/services` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:47.314349 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842b4636-1564-49d6-802f-ce4cafa324b6 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:47.314880 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842b4636-1564-49d6-802f-ce4cafa324b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:47.316424 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842b4636-1564-49d6-802f-ce4cafa324b6 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:47.326489 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 418/837] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Sep 19 16:00:47 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:47.893091 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7719ea-ed7f-4f47-9dbc-6ddce13156c0 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:47.893605 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7719ea-ed7f-4f47-9dbc-6ddce13156c0 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:47.893710 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7719ea-ed7f-4f47-9dbc-6ddce13156c0 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:47.895263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 420/838] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:00:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:00:47.900652 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45129a0e-d1d9-4bd2-87ed-620c8ea63cc7 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:47.901105 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45129a0e-d1d9-4bd2-87ed-620c8ea63cc7 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:47.901190 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45129a0e-d1d9-4bd2-87ed-620c8ea63cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:47.921023 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-45129a0e-d1d9-4bd2-87ed-620c8ea63cc7 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:00:47.922227 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-45129a0e-d1d9-4bd2-87ed-620c8ea63cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:00:48.000360 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 419/839] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:00:47 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 16:00:48.062361 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e5156-449f-4dec-bef2-cc64baa18e3e None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:48.062996 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e5156-449f-4dec-bef2-cc64baa18e3e None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:48.063489 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e5156-449f-4dec-bef2-cc64baa18e3e None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:48.065289 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 421/840] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:00:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:00:48.072033 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2929e9-98c0-4b1b-a7d3-b847468da0c5 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:48.072646 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2929e9-98c0-4b1b-a7d3-b847468da0c5 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:48.073199 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2929e9-98c0-4b1b-a7d3-b847468da0c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:48.094499 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-fc2929e9-98c0-4b1b-a7d3-b847468da0c5 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:00:48.096135 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-fc2929e9-98c0-4b1b-a7d3-b847468da0c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:00:48.098879 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:48.179096 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 420/841] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:00:48 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 16:00:48.275910 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e5156-449f-4dec-bef2-cc64baa18e3e None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:48.299188 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a07d479-0bdf-46f3-8276-574d6046001c None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:48.299514 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a07d479-0bdf-46f3-8276-574d6046001c None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:48.299739 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a07d479-0bdf-46f3-8276-574d6046001c None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:48.301302 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 421/842] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:00:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:00:48.306694 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d43c2c4-1797-4752-8a37-889ec1796fe4 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:48.306977 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d43c2c4-1797-4752-8a37-889ec1796fe4 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:48.307253 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d43c2c4-1797-4752-8a37-889ec1796fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:48.333576 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-9d43c2c4-1797-4752-8a37-889ec1796fe4 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:00:48.336222 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-9d43c2c4-1797-4752-8a37-889ec1796fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:00:48.338872 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:48.411009 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e5156-449f-4dec-bef2-cc64baa18e3e None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:48.441998 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 422/843] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:00:48 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 16:00:48.456471 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d43c2c4-1797-4752-8a37-889ec1796fe4 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:48.460555 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8f184f-d0ba-4b83-88b6-d003c532aa75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:48.461557 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f184f-d0ba-4b83-88b6-d003c532aa75 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:48.462063 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f184f-d0ba-4b83-88b6-d003c532aa75 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:48.462598 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f184f-d0ba-4b83-88b6-d003c532aa75 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:48.463429 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8f184f-d0ba-4b83-88b6-d003c532aa75 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:48.465113 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8f184f-d0ba-4b83-88b6-d003c532aa75 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:48.482963 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-2f8f184f-d0ba-4b83-88b6-d003c532aa75 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 19 16:00:48.482963 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:48.482963 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 16:00:48.482963 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 16:00:48.482963 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 16:00:48.482963 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 16:00:48.482963 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 16:00:48.482963 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 16:00:48.482963 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 19 16:00:48.482963 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:48.482963 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 16:00:48.482963 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:48.482963 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:48.482963 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:48.482963 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:48.482963 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:48.482963 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:48.484122 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:48.484122 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:48.484122 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:48.484122 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:48.484122 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:48.484122 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:48.484122 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 16:00:48.484122 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 16:00:48.484122 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 16:00:48.484122 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 16:00:48.484122 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:48.484122 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:48.484122 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:48.484122 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:48.484122 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:48.484122 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:48.485684 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:48.485684 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:48.485684 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:48.485684 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:48.485684 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:48.485684 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:48.485684 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:48.485684 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:48.485684 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:48.485684 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:48.485684 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 16:00:48.485684 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 16:00:48.485684 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 19 16:00:48.485684 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:48.494825 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 422/844] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Sep 19 16:00:48 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:48.496089 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8f184f-d0ba-4b83-88b6-d003c532aa75 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:48.549757 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:48.600773 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d43c2c4-1797-4752-8a37-889ec1796fe4 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:48.610654 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8f184f-d0ba-4b83-88b6-d003c532aa75 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:48.618482 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:48.640501 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b2ae12-e284-4245-9699-f142202c0970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:48.641419 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b2ae12-e284-4245-9699-f142202c0970 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:48.641878 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b2ae12-e284-4245-9699-f142202c0970 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:48.642333 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b2ae12-e284-4245-9699-f142202c0970 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:48.648698 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db457c50-65bb-4647-b3a2-96606ea60e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:48.649309 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db457c50-65bb-4647-b3a2-96606ea60e49 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:48.649589 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db457c50-65bb-4647-b3a2-96606ea60e49 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:48.649862 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db457c50-65bb-4647-b3a2-96606ea60e49 None admin] PATH_INFO: `/v3/services` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:48.650527 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db457c50-65bb-4647-b3a2-96606ea60e49 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:48.650839 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db457c50-65bb-4647-b3a2-96606ea60e49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:48.652205 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db457c50-65bb-4647-b3a2-96606ea60e49 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:48.657184 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b2ae12-e284-4245-9699-f142202c0970 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 16:00:48.661820 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 423/845] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Sep 19 16:00:48 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:48.662729 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db457c50-65bb-4647-b3a2-96606ea60e49 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:48.663012 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-a4b2ae12-e284-4245-9699-f142202c0970 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 16:00:48.663012 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:48.663012 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:48.663012 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:48.663012 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:48.663012 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:48.663012 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:48.663012 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:48.663012 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:48.663012 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:48.663012 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:48.663012 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:48.663012 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 16:00:48.663012 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 16:00:48.663012 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 16:00:48.663012 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 16:00:48.663908 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 16:00:48.663908 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 16:00:48.663908 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 16:00:48.663908 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 16:00:48.663908 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:48.663908 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:48.663908 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:48.663908 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:48.663908 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:48.663908 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:48.663908 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:48.663908 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:48.663908 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:48.663908 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:48.663908 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:48.663908 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:48.664725 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:48.664725 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:48.664725 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:48.664725 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:48.664725 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 16:00:48.664725 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 16:00:48.664725 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 16:00:48.664725 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 16:00:48.664725 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 16:00:48.664725 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 16:00:48.664725 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 16:00:48.664725 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:48.667640 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 423/846] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Sep 19 16:00:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 16:00:48.676122 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b2ae12-e284-4245-9699-f142202c0970 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:48.799617 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db457c50-65bb-4647-b3a2-96606ea60e49 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:48.804077 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b2ae12-e284-4245-9699-f142202c0970 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:48.831437 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:48.835880 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961a59b6-7f4c-41ff-bb14-f49c413ba37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:48.836487 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961a59b6-7f4c-41ff-bb14-f49c413ba37d None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:48.836853 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961a59b6-7f4c-41ff-bb14-f49c413ba37d None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:48.837123 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961a59b6-7f4c-41ff-bb14-f49c413ba37d None admin] PATH_INFO: `/v3/services` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:48.837739 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961a59b6-7f4c-41ff-bb14-f49c413ba37d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:48.838040 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961a59b6-7f4c-41ff-bb14-f49c413ba37d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:48.839638 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961a59b6-7f4c-41ff-bb14-f49c413ba37d None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:48.840730 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d583d640-2af4-42da-9e62-f6f3c145693d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:48.841514 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d583d640-2af4-42da-9e62-f6f3c145693d None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:48.841800 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d583d640-2af4-42da-9e62-f6f3c145693d None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:48.842413 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d583d640-2af4-42da-9e62-f6f3c145693d None admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:48.843073 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d583d640-2af4-42da-9e62-f6f3c145693d None admin] RBAC: Adding query filter params (name=service) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:48.843388 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d583d640-2af4-42da-9e62-f6f3c145693d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:48.844734 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d583d640-2af4-42da-9e62-f6f3c145693d None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:48.844992 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 424/847] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Sep 19 16:00:48 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:48.846075 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961a59b6-7f4c-41ff-bb14-f49c413ba37d None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:48.869288 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 424/848] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Sep 19 16:00:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:00:48.870578 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d583d640-2af4-42da-9e62-f6f3c145693d None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:48.896509 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:48.977667 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961a59b6-7f4c-41ff-bb14-f49c413ba37d None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:49.018736 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4233e44e-ddc9-425e-9468-3d1b28af8f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:49.019520 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4233e44e-ddc9-425e-9468-3d1b28af8f72 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:49.019907 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4233e44e-ddc9-425e-9468-3d1b28af8f72 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:49.020318 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4233e44e-ddc9-425e-9468-3d1b28af8f72 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:49.029113 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d583d640-2af4-42da-9e62-f6f3c145693d None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:49.033288 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4233e44e-ddc9-425e-9468-3d1b28af8f72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 16:00:49.038363 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-4233e44e-ddc9-425e-9468-3d1b28af8f72 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 16:00:49.038363 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:49.038363 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:49.038363 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:49.038363 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:49.038363 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:49.038363 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:49.038363 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:49.038363 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:49.038363 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:49.038363 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:49.038363 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:49.038363 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 16:00:49.038363 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 16:00:49.038363 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 16:00:49.038363 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 16:00:49.039396 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 16:00:49.039396 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 16:00:49.039396 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 16:00:49.039396 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 16:00:49.039396 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:49.039396 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:49.039396 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:49.039396 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:49.039396 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:49.039396 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:49.039396 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:49.039396 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:49.039396 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:49.039396 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:49.039396 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:49.039396 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:49.040230 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:49.040230 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:49.040230 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:49.040230 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:49.040230 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 16:00:49.040230 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 16:00:49.040230 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 16:00:49.040230 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 16:00:49.040230 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 16:00:49.040230 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 16:00:49.040230 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 16:00:49.040230 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:49.040230 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 425/849] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 19 16:00:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:49.050171 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4233e44e-ddc9-425e-9468-3d1b28af8f72 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:49.099516 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc5f166-a83e-47f4-894d-06ab852b523a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:49.100189 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc5f166-a83e-47f4-894d-06ab852b523a None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:49.100464 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc5f166-a83e-47f4-894d-06ab852b523a None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:49.100732 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc5f166-a83e-47f4-894d-06ab852b523a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:49.113598 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc5f166-a83e-47f4-894d-06ab852b523a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:49.115683 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc5f166-a83e-47f4-894d-06ab852b523a None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:49.132430 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-abc5f166-a83e-47f4-894d-06ab852b523a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:49.132430 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:49.132430 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:49.132430 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:49.132430 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:49.132430 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:49.132430 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:49.132430 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:49.132430 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:49.132430 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:49.132430 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:49.132430 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:49.132430 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:00:49.132430 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:00:49.132430 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:00:49.132430 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:00:49.133399 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:49.133399 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:49.133399 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:49.133399 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:49.133399 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:49.133399 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:49.133399 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:49.133399 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:49.133399 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:49.133399 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:49.133399 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:49.133399 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:49.133399 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:49.133399 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:49.133399 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:49.133399 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:49.133399 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:00:49.134317 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:00:49.134317 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:49.134317 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:49.134317 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 425/850] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:00:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:49.139540 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc5f166-a83e-47f4-894d-06ab852b523a None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:49.171965 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:49.181159 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4233e44e-ddc9-425e-9468-3d1b28af8f72 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:49.194883 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:49.221456 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424ca3aa-1baa-4507-9bf7-82db01cf1fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:49.222492 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424ca3aa-1baa-4507-9bf7-82db01cf1fa0 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:49.223014 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424ca3aa-1baa-4507-9bf7-82db01cf1fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:49.223489 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424ca3aa-1baa-4507-9bf7-82db01cf1fa0 None admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:49.224357 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424ca3aa-1baa-4507-9bf7-82db01cf1fa0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:49.224878 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424ca3aa-1baa-4507-9bf7-82db01cf1fa0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:49.226449 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424ca3aa-1baa-4507-9bf7-82db01cf1fa0 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:49.238342 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 426/851] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 16:00:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:00:49.243867 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424ca3aa-1baa-4507-9bf7-82db01cf1fa0 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:49.267470 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc5f166-a83e-47f4-894d-06ab852b523a None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:49.312817 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c687ddc4-b3d4-4ed8-8467-b21434fcac37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:49.313878 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c687ddc4-b3d4-4ed8-8467-b21434fcac37 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:49.314364 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c687ddc4-b3d4-4ed8-8467-b21434fcac37 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:49.314869 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c687ddc4-b3d4-4ed8-8467-b21434fcac37 None admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:49.315711 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c687ddc4-b3d4-4ed8-8467-b21434fcac37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:49.316209 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c687ddc4-b3d4-4ed8-8467-b21434fcac37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:49.317772 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c687ddc4-b3d4-4ed8-8467-b21434fcac37 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:49.335135 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 426/852] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:00:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:00:49.336367 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c687ddc4-b3d4-4ed8-8467-b21434fcac37 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:49.380867 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424ca3aa-1baa-4507-9bf7-82db01cf1fa0 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:49.435286 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a665f9-f2f0-475a-927a-c9e4e5ba062e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:49.436646 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a665f9-f2f0-475a-927a-c9e4e5ba062e None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:49.437081 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a665f9-f2f0-475a-927a-c9e4e5ba062e None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:49.437502 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a665f9-f2f0-475a-927a-c9e4e5ba062e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:49.451756 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a665f9-f2f0-475a-927a-c9e4e5ba062e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:49.453202 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a665f9-f2f0-475a-927a-c9e4e5ba062e None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:49.454774 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c687ddc4-b3d4-4ed8-8467-b21434fcac37 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:49.457366 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:49.471628 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-d3a665f9-f2f0-475a-927a-c9e4e5ba062e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:49.471628 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:49.471628 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:49.471628 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:49.471628 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:49.471628 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:49.471628 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:49.471628 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:49.471628 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:49.471628 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:49.471628 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:49.471628 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:49.471628 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:00:49.471628 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:00:49.471628 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:00:49.471628 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:00:49.472680 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:49.472680 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:49.472680 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:49.472680 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:49.472680 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:49.472680 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:49.472680 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:49.472680 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:49.472680 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:49.472680 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:49.472680 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:49.472680 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:49.472680 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:49.472680 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:49.472680 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:49.472680 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:49.472680 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:00:49.473558 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:00:49.473558 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:49.473558 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:49.474617 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 427/853] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:00:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:49.480097 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a665f9-f2f0-475a-927a-c9e4e5ba062e None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:49.493833 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:49.518632 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfb9e6a-8fb8-4d7e-a950-c47b67fc89e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:49.519655 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfb9e6a-8fb8-4d7e-a950-c47b67fc89e7 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:49.520152 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfb9e6a-8fb8-4d7e-a950-c47b67fc89e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:49.520805 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfb9e6a-8fb8-4d7e-a950-c47b67fc89e7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:49.533288 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfb9e6a-8fb8-4d7e-a950-c47b67fc89e7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:49.535766 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfb9e6a-8fb8-4d7e-a950-c47b67fc89e7 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:49.552981 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-0bfb9e6a-8fb8-4d7e-a950-c47b67fc89e7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 16:00:49.552981 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:49.552981 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:49.552981 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:49.552981 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:49.552981 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:49.552981 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:49.552981 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:49.552981 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:49.552981 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:49.552981 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:49.552981 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:49.552981 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 16:00:49.552981 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 16:00:49.552981 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 16:00:49.552981 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 16:00:49.553982 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:49.553982 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:49.553982 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 16:00:49.553982 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:00:49.553982 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 16:00:49.553982 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:00:49.553982 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:49.553982 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:49.553982 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:49.553982 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:49.553982 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:49.553982 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:49.553982 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:49.553982 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:49.553982 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:49.553982 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:49.553982 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:49.553982 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:49.554948 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:49.554948 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:49.554948 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 16:00:49.554948 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 16:00:49.554948 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 16:00:49.554948 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 16:00:49.554948 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 16:00:49.554948 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 16:00:49.554948 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 16:00:49.554948 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:49.562718 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 427/854] 10.222.0.57 () {56 vars in 1117 bytes} [Mon Sep 19 16:00:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:49.564332 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfb9e6a-8fb8-4d7e-a950-c47b67fc89e7 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:49.600589 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a665f9-f2f0-475a-927a-c9e4e5ba062e None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:49.635266 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93d0ede-efd0-42d9-9894-ec8cbda168a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:49.635950 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d0ede-efd0-42d9-9894-ec8cbda168a1 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:49.636226 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d0ede-efd0-42d9-9894-ec8cbda168a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:49.636517 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d0ede-efd0-42d9-9894-ec8cbda168a1 None admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:49.637688 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93d0ede-efd0-42d9-9894-ec8cbda168a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:49.637992 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93d0ede-efd0-42d9-9894-ec8cbda168a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:49.639381 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93d0ede-efd0-42d9-9894-ec8cbda168a1 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:49.655733 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 428/855] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:00:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:00:49.657160 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a8a37-31cb-4ff3-9c7d-2b1e08de3060 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:49.657450 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a8a37-31cb-4ff3-9c7d-2b1e08de3060 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:49.657723 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a8a37-31cb-4ff3-9c7d-2b1e08de3060 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:49.662991 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 429/856] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:00:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:00:49.662991 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6a8a37-31cb-4ff3-9c7d-2b1e08de3060 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:49.690664 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfb9e6a-8fb8-4d7e-a950-c47b67fc89e7 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:49.726050 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d13681a-a66d-4f2a-af75-5a4f79ce85d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:49.726697 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d13681a-a66d-4f2a-af75-5a4f79ce85d9 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:49.726957 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d13681a-a66d-4f2a-af75-5a4f79ce85d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:49.727242 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d13681a-a66d-4f2a-af75-5a4f79ce85d9 None admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:49.728107 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d13681a-a66d-4f2a-af75-5a4f79ce85d9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:49.728323 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d13681a-a66d-4f2a-af75-5a4f79ce85d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:49.729776 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d13681a-a66d-4f2a-af75-5a4f79ce85d9 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:49.737892 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:49.753131 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 428/857] 10.222.0.57 () {56 vars in 1149 bytes} [Mon Sep 19 16:00:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:00:49.753131 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd26561-29a0-4c32-96f5-53cefd056822 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:49.753131 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd26561-29a0-4c32-96f5-53cefd056822 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:49.753131 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd26561-29a0-4c32-96f5-53cefd056822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:49.777390 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-acd26561-29a0-4c32-96f5-53cefd056822 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:00:49.778813 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-acd26561-29a0-4c32-96f5-53cefd056822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:00:49.780206 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6a8a37-31cb-4ff3-9c7d-2b1e08de3060 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:49.795602 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:49.822721 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ca563-c6b0-4589-ae8c-45f2063382eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:49.823371 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ca563-c6b0-4589-ae8c-45f2063382eb None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:49.823650 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ca563-c6b0-4589-ae8c-45f2063382eb None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:49.823921 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ca563-c6b0-4589-ae8c-45f2063382eb None admin] PATH_INFO: `/v3/users/swift` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:49.832191 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38ca563-c6b0-4589-ae8c-45f2063382eb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:49.833654 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38ca563-c6b0-4589-ae8c-45f2063382eb None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:49.846810 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-f38ca563-c6b0-4589-ae8c-45f2063382eb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 19 16:00:49.846810 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:49.846810 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:49.846810 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:49.846810 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:49.846810 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:49.846810 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:49.846810 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:49.846810 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:49.846810 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:49.846810 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:49.846810 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:49.846810 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 16:00:49.846810 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 16:00:49.846810 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 16:00:49.846810 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 16:00:49.847836 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:49.847836 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:49.847836 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 16:00:49.847836 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:00:49.847836 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 16:00:49.847836 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:00:49.847836 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:49.847836 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:49.847836 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:49.847836 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:49.847836 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:49.847836 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:49.847836 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:49.847836 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:49.847836 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:49.847836 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:49.847836 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:49.847836 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:49.848771 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:49.848771 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:49.848771 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 16:00:49.848771 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 16:00:49.848771 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 16:00:49.848771 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 16:00:49.848771 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 16:00:49.848771 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 16:00:49.848771 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 19 16:00:49.848771 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:49.848771 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 430/858] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 19 16:00:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:49.848771 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ca563-c6b0-4589-ae8c-45f2063382eb None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:49.870584 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 429/859] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:00:49 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 16:00:49.870584 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd26561-29a0-4c32-96f5-53cefd056822 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:49.961591 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ca563-c6b0-4589-ae8c-45f2063382eb None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:49.974325 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd26561-29a0-4c32-96f5-53cefd056822 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:49.990591 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:49.999568 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888db490-e649-4268-a0a1-17c27c355b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:50.000185 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888db490-e649-4268-a0a1-17c27c355b21 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:50.000495 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888db490-e649-4268-a0a1-17c27c355b21 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:50.000776 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888db490-e649-4268-a0a1-17c27c355b21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:50.014129 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc37c96-2b26-4d0b-aedd-a618b48e37cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:50.015002 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc37c96-2b26-4d0b-aedd-a618b48e37cb None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:50.015438 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc37c96-2b26-4d0b-aedd-a618b48e37cb None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:50.015851 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc37c96-2b26-4d0b-aedd-a618b48e37cb None admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:50.016758 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc37c96-2b26-4d0b-aedd-a618b48e37cb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:50.017265 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc37c96-2b26-4d0b-aedd-a618b48e37cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:50.018971 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc37c96-2b26-4d0b-aedd-a618b48e37cb None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:50.046137 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 430/860] 10.222.0.57 () {56 vars in 1135 bytes} [Mon Sep 19 16:00:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:50.051545 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc37c96-2b26-4d0b-aedd-a618b48e37cb None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:50.062452 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:50.084152 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888db490-e649-4268-a0a1-17c27c355b21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:50.085779 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888db490-e649-4268-a0a1-17c27c355b21 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:50.162843 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc37c96-2b26-4d0b-aedd-a618b48e37cb None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:50.199220 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 431/861] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 19 16:00:49 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 16:00:50.200756 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888db490-e649-4268-a0a1-17c27c355b21 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:50.204547 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dc806c-77d9-42f4-a477-b488fc68b4a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:50.205175 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dc806c-77d9-42f4-a477-b488fc68b4a0 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:50.205461 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dc806c-77d9-42f4-a477-b488fc68b4a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:50.205730 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dc806c-77d9-42f4-a477-b488fc68b4a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:50.232005 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:50.276742 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dc806c-77d9-42f4-a477-b488fc68b4a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:50.278518 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dc806c-77d9-42f4-a477-b488fc68b4a0 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:50.288949 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:50.313645 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888db490-e649-4268-a0a1-17c27c355b21 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:50.357237 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e900b476-be14-45f3-a1f0-50048b94e930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:50.358141 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900b476-be14-45f3-a1f0-50048b94e930 None admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:50.358721 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900b476-be14-45f3-a1f0-50048b94e930 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:50.359000 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900b476-be14-45f3-a1f0-50048b94e930 None admin] PATH_INFO: `/v3/services` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:50.359650 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e900b476-be14-45f3-a1f0-50048b94e930 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:50.361547 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e900b476-be14-45f3-a1f0-50048b94e930 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:50.378606 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 432/862] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Sep 19 16:00:50 2022] POST /identity/v3/services => generated 269 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:00:50.379731 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e900b476-be14-45f3-a1f0-50048b94e930 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:50.380790 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 431/863] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 19 16:00:50 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 19 16:00:50.387502 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dc806c-77d9-42f4-a477-b488fc68b4a0 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:50.513038 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:50.522910 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dc806c-77d9-42f4-a477-b488fc68b4a0 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:50.534212 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e900b476-be14-45f3-a1f0-50048b94e930 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:50.536106 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:50.570224 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1c1168-53ff-4d9d-bd2d-a933faa48c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:50.571692 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1c1168-53ff-4d9d-bd2d-a933faa48c4a None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:50.572161 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1c1168-53ff-4d9d-bd2d-a933faa48c4a None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:50.573704 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1c1168-53ff-4d9d-bd2d-a933faa48c4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:50.586342 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e4aacc-08e6-494f-ae1a-a79c56760153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:50.587290 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e4aacc-08e6-494f-ae1a-a79c56760153 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:50.587573 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e4aacc-08e6-494f-ae1a-a79c56760153 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:50.587821 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1c1168-53ff-4d9d-bd2d-a933faa48c4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:50.587937 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e4aacc-08e6-494f-ae1a-a79c56760153 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:50.589288 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1c1168-53ff-4d9d-bd2d-a933faa48c4a None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:50.602557 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e4aacc-08e6-494f-ae1a-a79c56760153 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:50.604101 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e4aacc-08e6-494f-ae1a-a79c56760153 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:50.608502 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-7b1c1168-53ff-4d9d-bd2d-a933faa48c4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:50.608502 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:50.608502 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:50.608502 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:50.608502 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:50.608502 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:50.608502 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:50.608502 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:50.608502 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:50.608502 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:50.608502 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:50.608502 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:50.608502 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:00:50.608502 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:00:50.608502 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:00:50.608502 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:00:50.609438 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:50.609438 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:50.609438 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:50.609438 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:50.609438 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:50.609438 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:50.609438 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:50.609438 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:50.609438 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:50.609438 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:50.609438 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:50.609438 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:50.609438 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:50.609438 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:50.609438 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:50.609438 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:50.609438 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:00:50.610316 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:00:50.610316 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:50.610316 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:50.612229 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 432/864] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:00:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 16:00:50.617859 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1c1168-53ff-4d9d-bd2d-a933faa48c4a None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:50.623189 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-01e4aacc-08e6-494f-ae1a-a79c56760153 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:50.623189 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:50.623189 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:50.623189 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:50.623189 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:50.623189 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:50.623189 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:50.623189 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:50.623189 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:50.623189 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:50.623189 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:50.623189 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:50.623189 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:00:50.623189 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:00:50.623189 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:00:50.623189 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:00:50.624206 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:50.624206 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:50.624206 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:50.624206 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:50.624206 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:50.624206 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:50.624206 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:50.624206 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:50.624206 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:50.624206 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:50.624206 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:50.624206 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:50.624206 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:50.624206 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:50.624206 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:50.624206 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:50.624206 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:00:50.625096 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:00:50.625096 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:50.625096 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:50.625096 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 433/865] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:00:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:50.630535 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e4aacc-08e6-494f-ae1a-a79c56760153 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:50.755194 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1c1168-53ff-4d9d-bd2d-a933faa48c4a None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:50.782001 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e4aacc-08e6-494f-ae1a-a79c56760153 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:50.811436 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c0c02d-a048-4cf3-a3a4-7cb569578901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:50.812424 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c0c02d-a048-4cf3-a3a4-7cb569578901 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:50.813170 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c0c02d-a048-4cf3-a3a4-7cb569578901 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:50.813666 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c0c02d-a048-4cf3-a3a4-7cb569578901 None admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:50.815063 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c0c02d-a048-4cf3-a3a4-7cb569578901 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:50.815612 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c0c02d-a048-4cf3-a3a4-7cb569578901 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:50.817888 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c0c02d-a048-4cf3-a3a4-7cb569578901 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:50.828670 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d7fac7-3fbc-44bb-ae05-80f3a5073b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:50.829307 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7fac7-3fbc-44bb-ae05-80f3a5073b6d None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:50.829589 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7fac7-3fbc-44bb-ae05-80f3a5073b6d None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:50.829903 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7fac7-3fbc-44bb-ae05-80f3a5073b6d None admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:50.830591 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d7fac7-3fbc-44bb-ae05-80f3a5073b6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:50.830900 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d7fac7-3fbc-44bb-ae05-80f3a5073b6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:50.832462 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d7fac7-3fbc-44bb-ae05-80f3a5073b6d None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:50.835614 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:50.837266 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 433/866] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:00:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:00:50.846446 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c0c02d-a048-4cf3-a3a4-7cb569578901 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:50.860851 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 434/867] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:00:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:50.862759 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d7fac7-3fbc-44bb-ae05-80f3a5073b6d None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:50.877367 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:50.982526 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c0c02d-a048-4cf3-a3a4-7cb569578901 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:50.994274 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d7fac7-3fbc-44bb-ae05-80f3a5073b6d None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:51.029013 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421a0196-5888-4732-ac8a-2f2cf1a2d970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:51.030017 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a0196-5888-4732-ac8a-2f2cf1a2d970 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:51.030577 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a0196-5888-4732-ac8a-2f2cf1a2d970 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:51.031061 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a0196-5888-4732-ac8a-2f2cf1a2d970 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:51.038542 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534876a9-ba49-4fab-9756-2a1c9f586508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:51.039189 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534876a9-ba49-4fab-9756-2a1c9f586508 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:51.039466 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534876a9-ba49-4fab-9756-2a1c9f586508 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:51.039739 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534876a9-ba49-4fab-9756-2a1c9f586508 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:51.040134 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421a0196-5888-4732-ac8a-2f2cf1a2d970 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:51.042311 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421a0196-5888-4732-ac8a-2f2cf1a2d970 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:51.048499 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534876a9-ba49-4fab-9756-2a1c9f586508 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:51.050318 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534876a9-ba49-4fab-9756-2a1c9f586508 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:51.056349 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-421a0196-5888-4732-ac8a-2f2cf1a2d970 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:00:51.056349 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:51.056349 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:51.056349 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:51.056349 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:51.056349 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:51.056349 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:51.056349 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:51.056349 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:51.056349 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:51.056349 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:51.056349 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:51.056349 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 16:00:51.056349 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 16:00:51.056349 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 16:00:51.057579 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 16:00:51.057579 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:51.057579 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:51.057579 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:51.057579 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:51.057579 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:51.057579 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:51.057579 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:51.057579 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:51.057579 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:51.057579 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:51.057579 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:51.057579 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:51.057579 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:51.057579 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:51.057579 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:51.057579 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:51.058503 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 16:00:51.058503 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 16:00:51.058503 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 16:00:51.058503 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 16:00:51.058503 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 16:00:51.058503 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 16:00:51.058503 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:00:51.058503 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:51.061639 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 434/868] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 19 16:00:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:51.068589 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-534876a9-ba49-4fab-9756-2a1c9f586508 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:00:51.068589 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:51.068589 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:51.068589 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:51.068589 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:51.068589 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:51.068589 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:51.068589 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:51.068589 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:51.068589 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:51.068589 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:51.068589 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:51.068589 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 16:00:51.068589 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 16:00:51.068589 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 16:00:51.069553 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 16:00:51.069553 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:51.069553 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:51.069553 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:51.069553 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:51.069553 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:51.069553 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:51.069553 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:51.069553 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:51.069553 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:51.069553 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:51.069553 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:51.069553 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:51.069553 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:51.069553 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:51.069553 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:51.069553 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:51.070881 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 16:00:51.070881 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 16:00:51.070881 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 16:00:51.070881 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 16:00:51.070881 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 16:00:51.070881 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 16:00:51.070881 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:00:51.070881 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:51.075045 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 435/869] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 19 16:00:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:51.078485 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421a0196-5888-4732-ac8a-2f2cf1a2d970 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:51.078670 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534876a9-ba49-4fab-9756-2a1c9f586508 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:51.150125 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:51.182430 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:51.204665 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534876a9-ba49-4fab-9756-2a1c9f586508 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:51.204863 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421a0196-5888-4732-ac8a-2f2cf1a2d970 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:51.239168 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6f6150-584a-4b41-965e-e06c9baf29c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:51.239785 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f6150-584a-4b41-965e-e06c9baf29c3 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:51.240063 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f6150-584a-4b41-965e-e06c9baf29c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:51.240265 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df46c786-3716-48b1-a0d3-4a499003f7eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:51.240389 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f6150-584a-4b41-965e-e06c9baf29c3 None admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:51.240836 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df46c786-3716-48b1-a0d3-4a499003f7eb None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:51.241015 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6f6150-584a-4b41-965e-e06c9baf29c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:51.241112 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df46c786-3716-48b1-a0d3-4a499003f7eb None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:51.241321 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6f6150-584a-4b41-965e-e06c9baf29c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:51.241427 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df46c786-3716-48b1-a0d3-4a499003f7eb None admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:51.242054 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df46c786-3716-48b1-a0d3-4a499003f7eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:51.242362 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df46c786-3716-48b1-a0d3-4a499003f7eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:51.242776 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6f6150-584a-4b41-965e-e06c9baf29c3 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:51.243796 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df46c786-3716-48b1-a0d3-4a499003f7eb None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:51.256015 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 435/870] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 19 16:00:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:51.257252 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06370e58-8028-49f4-9f45-0ce29223ea8d None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:51.257538 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06370e58-8028-49f4-9f45-0ce29223ea8d None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:51.257861 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06370e58-8028-49f4-9f45-0ce29223ea8d None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:51.262988 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 436/871] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:00:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:00:51.262988 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06370e58-8028-49f4-9f45-0ce29223ea8d None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:51.266451 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 436/872] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 19 16:00:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:00:51.267948 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db089e-43fe-40f8-b76e-de6c7c74a819 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:51.268351 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db089e-43fe-40f8-b76e-de6c7c74a819 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:51.268632 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db089e-43fe-40f8-b76e-de6c7c74a819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:51.288098 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-38db089e-43fe-40f8-b76e-de6c7c74a819 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:00:51.290399 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-38db089e-43fe-40f8-b76e-de6c7c74a819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:00:51.364604 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 437/873] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:00:51 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 16:00:51.365734 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38db089e-43fe-40f8-b76e-de6c7c74a819 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:51.378590 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06370e58-8028-49f4-9f45-0ce29223ea8d None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:51.424582 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1efbf1-5729-44dc-9834-d8f95120f24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:51.425373 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1efbf1-5729-44dc-9834-d8f95120f24e None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:51.425720 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1efbf1-5729-44dc-9834-d8f95120f24e None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:51.426098 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1efbf1-5729-44dc-9834-d8f95120f24e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:51.426928 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1efbf1-5729-44dc-9834-d8f95120f24e None admin] RBAC: Adding query filter params (role.id=63d2d197bfd6497583e94bfb065ce29b, scope.project.id=be03e5300b3343169f35c643c93d1e06, user.id=884a0524aca744b99c7496c11c3291ae) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:51.427304 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1efbf1-5729-44dc-9834-d8f95120f24e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:51.428854 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1efbf1-5729-44dc-9834-d8f95120f24e None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:51.438760 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:51.442888 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 437/874] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 19 16:00:51 2022] GET /identity/v3/role_assignments?user.id=884a0524aca744b99c7496c11c3291ae&scope.project.id=be03e5300b3343169f35c643c93d1e06&role.id=63d2d197bfd6497583e94bfb065ce29b => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:51.525251 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38db089e-43fe-40f8-b76e-de6c7c74a819 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:51.573820 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3fdc09-b36a-44d1-8b71-62bb907e1ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:51.575432 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3fdc09-b36a-44d1-8b71-62bb907e1ff3 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:51.575931 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3fdc09-b36a-44d1-8b71-62bb907e1ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:51.576424 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3fdc09-b36a-44d1-8b71-62bb907e1ff3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:51.577401 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3fdc09-b36a-44d1-8b71-62bb907e1ff3 None admin] RBAC: Adding query filter params (role.id=1e475e6a170a4dbabc4ce8348eeb30b9, scope.project.id=be03e5300b3343169f35c643c93d1e06, user.id=7222bd3b313b47e5a4fa0ebc9ff0cdd5) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:51.577929 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3fdc09-b36a-44d1-8b71-62bb907e1ff3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:51.579720 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3fdc09-b36a-44d1-8b71-62bb907e1ff3 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:51.587426 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 438/875] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 19 16:00:51 2022] GET /identity/v3/role_assignments?user.id=7222bd3b313b47e5a4fa0ebc9ff0cdd5&scope.project.id=be03e5300b3343169f35c643c93d1e06&role.id=1e475e6a170a4dbabc4ce8348eeb30b9 => generated 608 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:51.694754 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1efbf1-5729-44dc-9834-d8f95120f24e None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:51.701088 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:51.843041 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1efbf1-5729-44dc-9834-d8f95120f24e None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:51.888175 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a61a8ee-d1b4-4977-a7ae-e0fce4f9765c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:51.889233 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a61a8ee-d1b4-4977-a7ae-e0fce4f9765c None admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:51.889721 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a61a8ee-d1b4-4977-a7ae-e0fce4f9765c None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:51.890229 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a61a8ee-d1b4-4977-a7ae-e0fce4f9765c None admin] PATH_INFO: `/v3/services` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:51.891149 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a61a8ee-d1b4-4977-a7ae-e0fce4f9765c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:51.892736 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a61a8ee-d1b4-4977-a7ae-e0fce4f9765c None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:51.913789 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 438/876] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Sep 19 16:00:51 2022] POST /identity/v3/services => generated 251 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:00:52.701111 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3839ec-1065-4e6a-9c82-31cedcad7524 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:52.701663 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3839ec-1065-4e6a-9c82-31cedcad7524 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:52.701663 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3839ec-1065-4e6a-9c82-31cedcad7524 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:52.703285 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 439/877] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:00:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:00:52.708691 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b2e3d6-87c3-4496-b0a1-e5f02d411fd1 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:52.709133 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b2e3d6-87c3-4496-b0a1-e5f02d411fd1 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:52.709231 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b2e3d6-87c3-4496-b0a1-e5f02d411fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:52.728226 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-11b2e3d6-87c3-4496-b0a1-e5f02d411fd1 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:00:52.729422 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-11b2e3d6-87c3-4496-b0a1-e5f02d411fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:00:52.796977 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:52.813385 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 439/878] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:00:52 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 16:00:53.086396 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3839ec-1065-4e6a-9c82-31cedcad7524 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:53.211098 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3839ec-1065-4e6a-9c82-31cedcad7524 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:53.223534 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:53.251066 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d553cf-f3d9-4373-bd73-d9581f9118ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:53.251718 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d553cf-f3d9-4373-bd73-d9581f9118ae None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:53.251999 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d553cf-f3d9-4373-bd73-d9581f9118ae None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:53.252275 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d553cf-f3d9-4373-bd73-d9581f9118ae None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:53.252877 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d553cf-f3d9-4373-bd73-d9581f9118ae None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:53.254253 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d553cf-f3d9-4373-bd73-d9581f9118ae None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:53.264058 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-63d553cf-f3d9-4373-bd73-d9581f9118ae None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 19 16:00:53.264058 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:53.264058 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 16:00:53.264058 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 16:00:53.264058 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 16:00:53.264058 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 16:00:53.264058 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 16:00:53.264058 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 16:00:53.264058 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 19 16:00:53.264058 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:53.264058 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 16:00:53.264058 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:53.264058 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:53.264058 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:53.264058 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:53.264058 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:53.265062 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:53.265062 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:53.265062 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:53.265062 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:53.265062 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:53.265062 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:53.265062 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:53.265062 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 16:00:53.265062 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 16:00:53.265062 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 16:00:53.265062 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 16:00:53.265062 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:53.265062 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:53.265062 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:53.265062 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:53.265062 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:53.265062 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:53.266590 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:53.266590 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:53.266590 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:53.266590 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:53.266590 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:53.266590 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:53.266590 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:53.266590 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:53.266590 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:53.266590 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:53.266590 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 16:00:53.266590 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 16:00:53.266590 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 19 16:00:53.266590 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:53.266590 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 440/879] 10.222.0.57 () {56 vars in 1132 bytes} [Mon Sep 19 16:00:53 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:53.271222 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b2e3d6-87c3-4496-b0a1-e5f02d411fd1 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:53.385108 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b2e3d6-87c3-4496-b0a1-e5f02d411fd1 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:53.422081 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f488e4-0926-4343-b3db-032d25c88e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:53.422747 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f488e4-0926-4343-b3db-032d25c88e05 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:53.423030 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f488e4-0926-4343-b3db-032d25c88e05 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:53.423308 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f488e4-0926-4343-b3db-032d25c88e05 None admin] PATH_INFO: `/v3/services` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:53.423952 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f488e4-0926-4343-b3db-032d25c88e05 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:53.424259 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f488e4-0926-4343-b3db-032d25c88e05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:53.425707 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f488e4-0926-4343-b3db-032d25c88e05 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:53.431326 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 440/880] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Sep 19 16:00:53 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:00:53.439761 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d553cf-f3d9-4373-bd73-d9581f9118ae None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:53.547913 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d553cf-f3d9-4373-bd73-d9581f9118ae None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:53.583029 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e1e8f-d389-4198-a4c8-dc35a4e30375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:53.583654 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e1e8f-d389-4198-a4c8-dc35a4e30375 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:53.583938 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e1e8f-d389-4198-a4c8-dc35a4e30375 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:53.584214 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e1e8f-d389-4198-a4c8-dc35a4e30375 None admin] PATH_INFO: `/v3/services` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:53.584864 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7e1e8f-d389-4198-a4c8-dc35a4e30375 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:53.585171 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7e1e8f-d389-4198-a4c8-dc35a4e30375 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:53.586587 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7e1e8f-d389-4198-a4c8-dc35a4e30375 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:53.593061 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 441/881] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Sep 19 16:00:53 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:53.599643 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f488e4-0926-4343-b3db-032d25c88e05 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:53.665432 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:53.711554 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd999f-f5e4-49fc-bb87-7fcb95bbdfdc None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:53.711554 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd999f-f5e4-49fc-bb87-7fcb95bbdfdc None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:53.715251 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd999f-f5e4-49fc-bb87-7fcb95bbdfdc None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:53.715251 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 442/882] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:00:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:00:53.722364 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55ca4b-4a45-449a-8961-8114f6bf1e95 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:53.722364 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55ca4b-4a45-449a-8961-8114f6bf1e95 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:53.722364 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55ca4b-4a45-449a-8961-8114f6bf1e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:53.722956 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f488e4-0926-4343-b3db-032d25c88e05 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:53.744359 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-cd55ca4b-4a45-449a-8961-8114f6bf1e95 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:00:53.747094 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-cd55ca4b-4a45-449a-8961-8114f6bf1e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:00:53.763938 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba61d40-f334-4136-8330-c39eda2675c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:53.764566 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba61d40-f334-4136-8330-c39eda2675c6 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:53.764895 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba61d40-f334-4136-8330-c39eda2675c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:53.765178 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba61d40-f334-4136-8330-c39eda2675c6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:53.766156 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba61d40-f334-4136-8330-c39eda2675c6 None admin] RBAC: Adding query filter params (interface=public, service_id=55758ff40d90457baf7a624bff18938b, region_id=RegionOne) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:53.766529 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba61d40-f334-4136-8330-c39eda2675c6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:53.768526 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba61d40-f334-4136-8330-c39eda2675c6 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:53.776118 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 441/883] 10.222.0.57 () {56 vars in 1251 bytes} [Mon Sep 19 16:00:53 2022] GET /identity/v3/endpoints?service_id=55758ff40d90457baf7a624bff18938b&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:53.788123 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba61d40-f334-4136-8330-c39eda2675c6 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:53.790854 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:53.847972 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 443/884] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:00:53 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 16:00:53.924714 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba61d40-f334-4136-8330-c39eda2675c6 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:53.943113 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:53.961531 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807904f7-48d3-4969-ab78-f90d0a963569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:53.962532 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807904f7-48d3-4969-ab78-f90d0a963569 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:53.963036 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807904f7-48d3-4969-ab78-f90d0a963569 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:53.963522 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807904f7-48d3-4969-ab78-f90d0a963569 None admin] PATH_INFO: `/v3/services` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:53.964320 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807904f7-48d3-4969-ab78-f90d0a963569 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:53.966438 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807904f7-48d3-4969-ab78-f90d0a963569 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:53.973022 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 442/885] 10.222.0.57 () {56 vars in 1087 bytes} [Mon Sep 19 16:00:53 2022] GET /identity/v3/services => generated 1268 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:00:53.992263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67872857-e2fe-407f-b6ea-c65b55443b8c None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:53.992621 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67872857-e2fe-407f-b6ea-c65b55443b8c None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:53.992824 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67872857-e2fe-407f-b6ea-c65b55443b8c None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:53.994671 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 444/886] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:00:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:00:54.001103 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0c39f1-b9cf-43fa-a98b-9e2e80df4968 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:54.001423 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0c39f1-b9cf-43fa-a98b-9e2e80df4968 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:54.001668 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0c39f1-b9cf-43fa-a98b-9e2e80df4968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:54.023085 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-ff0c39f1-b9cf-43fa-a98b-9e2e80df4968 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:00:54.024371 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-ff0c39f1-b9cf-43fa-a98b-9e2e80df4968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:00:54.110898 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 443/887] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:00:53 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 16:00:54.117857 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67872857-e2fe-407f-b6ea-c65b55443b8c None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:54.238235 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67872857-e2fe-407f-b6ea-c65b55443b8c None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:54.273345 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e567f523-f1fb-4996-8ac5-3debc32c8c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:54.273973 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e567f523-f1fb-4996-8ac5-3debc32c8c16 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:54.274246 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e567f523-f1fb-4996-8ac5-3debc32c8c16 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:54.274543 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e567f523-f1fb-4996-8ac5-3debc32c8c16 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:54.275123 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e567f523-f1fb-4996-8ac5-3debc32c8c16 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:54.276467 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e567f523-f1fb-4996-8ac5-3debc32c8c16 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:54.285490 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-e567f523-f1fb-4996-8ac5-3debc32c8c16 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 16:00:54.285490 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:54.285490 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 16:00:54.285490 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 16:00:54.285490 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 16:00:54.285490 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 16:00:54.285490 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 16:00:54.285490 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 16:00:54.285490 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 16:00:54.285490 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:54.285490 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 16:00:54.285490 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:54.285490 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:54.285490 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:54.285490 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:54.285490 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:54.285490 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:54.286529 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:54.286529 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:54.286529 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:54.286529 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:54.286529 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:54.286529 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:54.286529 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 16:00:54.286529 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 16:00:54.286529 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 16:00:54.286529 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 16:00:54.286529 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:54.286529 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:54.286529 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:54.286529 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:54.286529 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:54.286529 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:54.287511 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:54.287511 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:54.287511 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:54.287511 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:54.287511 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:54.287511 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:54.287511 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:54.287511 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:54.287511 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:54.287511 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:54.287511 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 16:00:54.287511 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 16:00:54.287511 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 16:00:54.287511 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:54.287511 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 445/888] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Sep 19 16:00:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:54.292892 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0c39f1-b9cf-43fa-a98b-9e2e80df4968 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:54.319070 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982acf33-f210-4494-ba2e-6ffd2043ce00 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:54.319346 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982acf33-f210-4494-ba2e-6ffd2043ce00 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:54.319821 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982acf33-f210-4494-ba2e-6ffd2043ce00 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:54.321745 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 446/889] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:00:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:00:54.327915 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7325c85-05dd-4419-b9b4-552fd1b08cde None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:54.328605 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7325c85-05dd-4419-b9b4-552fd1b08cde None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:54.329072 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7325c85-05dd-4419-b9b4-552fd1b08cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:54.348866 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-a7325c85-05dd-4419-b9b4-552fd1b08cde None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:00:54.350402 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-a7325c85-05dd-4419-b9b4-552fd1b08cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:00:54.380863 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:54.426000 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 447/890] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:00:54 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 16:00:54.427257 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7325c85-05dd-4419-b9b4-552fd1b08cde None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:54.432245 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0c39f1-b9cf-43fa-a98b-9e2e80df4968 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:54.456381 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:54.495364 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500b5966-8b01-4b2a-926f-332cf0865ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:54.497087 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b5966-8b01-4b2a-926f-332cf0865ea9 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:54.497582 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b5966-8b01-4b2a-926f-332cf0865ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:54.498063 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b5966-8b01-4b2a-926f-332cf0865ea9 None admin] PATH_INFO: `/v3/services` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:54.499324 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500b5966-8b01-4b2a-926f-332cf0865ea9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:54.499894 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500b5966-8b01-4b2a-926f-332cf0865ea9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:54.502774 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500b5966-8b01-4b2a-926f-332cf0865ea9 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:54.516297 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 444/891] 10.222.0.57 () {56 vars in 1122 bytes} [Mon Sep 19 16:00:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:54.519330 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500b5966-8b01-4b2a-926f-332cf0865ea9 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:54.575745 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7325c85-05dd-4419-b9b4-552fd1b08cde None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:54.611466 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4653b5-568c-45c9-97e0-5f5b31b41811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:54.612095 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4653b5-568c-45c9-97e0-5f5b31b41811 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:54.612370 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4653b5-568c-45c9-97e0-5f5b31b41811 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:54.612642 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4653b5-568c-45c9-97e0-5f5b31b41811 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:54.624279 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4653b5-568c-45c9-97e0-5f5b31b41811 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 16:00:54.629345 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-8e4653b5-568c-45c9-97e0-5f5b31b41811 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 16:00:54.629345 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:54.629345 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:54.629345 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:54.629345 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:54.629345 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:54.629345 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:54.629345 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:54.629345 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:54.629345 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:54.629345 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:54.629345 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:54.629345 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 16:00:54.629345 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 16:00:54.629345 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 16:00:54.629345 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 16:00:54.630412 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 16:00:54.630412 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 16:00:54.630412 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 16:00:54.630412 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 16:00:54.630412 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:54.630412 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:54.630412 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:54.630412 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:54.630412 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:54.630412 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:54.630412 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:54.630412 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:54.630412 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:54.630412 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:54.630412 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:54.630412 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:54.631435 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:54.631435 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:54.631435 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:54.631435 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:54.631435 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 16:00:54.631435 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 16:00:54.631435 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 16:00:54.631435 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 16:00:54.631435 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 16:00:54.631435 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 16:00:54.631435 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 16:00:54.631435 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:54.631435 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 448/892] 10.222.0.57 () {56 vars in 1120 bytes} [Mon Sep 19 16:00:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:54.638486 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4653b5-568c-45c9-97e0-5f5b31b41811 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:54.651027 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500b5966-8b01-4b2a-926f-332cf0865ea9 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:54.665786 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:54.692543 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd48b03c-5f90-47ed-b3d6-8e381455ee52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:54.693489 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd48b03c-5f90-47ed-b3d6-8e381455ee52 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:54.693954 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd48b03c-5f90-47ed-b3d6-8e381455ee52 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:54.694514 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd48b03c-5f90-47ed-b3d6-8e381455ee52 None admin] PATH_INFO: `/v3/services` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:54.695420 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd48b03c-5f90-47ed-b3d6-8e381455ee52 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:54.695897 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd48b03c-5f90-47ed-b3d6-8e381455ee52 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:54.697574 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd48b03c-5f90-47ed-b3d6-8e381455ee52 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:54.704721 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 445/893] 10.222.0.57 () {56 vars in 1122 bytes} [Mon Sep 19 16:00:54 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:54.723977 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd48b03c-5f90-47ed-b3d6-8e381455ee52 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:54.772512 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4653b5-568c-45c9-97e0-5f5b31b41811 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:54.781929 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:54.828074 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b0afab-dae2-4cc0-bbdc-1d1e49cea617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:54.828710 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0afab-dae2-4cc0-bbdc-1d1e49cea617 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:54.828992 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0afab-dae2-4cc0-bbdc-1d1e49cea617 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:54.829272 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0afab-dae2-4cc0-bbdc-1d1e49cea617 None admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:54.829946 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b0afab-dae2-4cc0-bbdc-1d1e49cea617 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:54.830253 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b0afab-dae2-4cc0-bbdc-1d1e49cea617 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:54.832445 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b0afab-dae2-4cc0-bbdc-1d1e49cea617 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:54.846476 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 449/894] 10.222.0.57 () {56 vars in 1115 bytes} [Mon Sep 19 16:00:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:54.850654 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b0afab-dae2-4cc0-bbdc-1d1e49cea617 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:54.862797 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd48b03c-5f90-47ed-b3d6-8e381455ee52 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:54.902840 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6145aa-b5f6-4d88-91a6-6df666e13709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:54.903607 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6145aa-b5f6-4d88-91a6-6df666e13709 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:54.903892 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6145aa-b5f6-4d88-91a6-6df666e13709 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:54.904282 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6145aa-b5f6-4d88-91a6-6df666e13709 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:54.905024 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6145aa-b5f6-4d88-91a6-6df666e13709 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:54.906585 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6145aa-b5f6-4d88-91a6-6df666e13709 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:54.917914 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-fe6145aa-b5f6-4d88-91a6-6df666e13709 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 19 16:00:54.917914 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:54.917914 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 16:00:54.917914 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 16:00:54.917914 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 16:00:54.917914 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 16:00:54.917914 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 16:00:54.917914 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 16:00:54.917914 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 19 16:00:54.917914 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:54.917914 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 16:00:54.917914 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:54.917914 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:54.917914 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:54.917914 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:54.917914 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:54.917914 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:54.919482 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:54.919482 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:54.919482 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:54.919482 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:54.919482 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:54.919482 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:54.919482 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 16:00:54.919482 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 16:00:54.919482 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 16:00:54.919482 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 16:00:54.919482 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:54.919482 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:54.919482 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:54.919482 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:54.919482 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:54.919482 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:54.920332 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:54.920332 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:54.920332 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:54.920332 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:54.920332 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:54.920332 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:54.920332 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:54.920332 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:54.920332 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:54.920332 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:54.920332 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 16:00:54.920332 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 16:00:54.920332 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 19 16:00:54.920332 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:54.920332 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 446/895] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Sep 19 16:00:54 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 16:00:54.926684 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6145aa-b5f6-4d88-91a6-6df666e13709 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:54.962507 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:54.983152 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b0afab-dae2-4cc0-bbdc-1d1e49cea617 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:55.018786 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d787f58f-c519-4d17-99b0-0bfe984d3712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:55.019416 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d787f58f-c519-4d17-99b0-0bfe984d3712 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:55.019692 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d787f58f-c519-4d17-99b0-0bfe984d3712 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:55.019964 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d787f58f-c519-4d17-99b0-0bfe984d3712 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:55.032817 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d787f58f-c519-4d17-99b0-0bfe984d3712 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:55.034236 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d787f58f-c519-4d17-99b0-0bfe984d3712 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:55.047724 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6145aa-b5f6-4d88-91a6-6df666e13709 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:55.050808 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-d787f58f-c519-4d17-99b0-0bfe984d3712 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:55.050808 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:55.050808 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:55.050808 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:55.050808 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:55.050808 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:55.050808 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:55.050808 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:55.050808 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:55.050808 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:55.050808 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:55.050808 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:55.050808 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:00:55.050808 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:00:55.050808 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:00:55.050808 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:00:55.051807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:55.051807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:55.051807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:55.051807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:55.051807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:55.051807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:55.051807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:55.051807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:55.051807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:55.051807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:55.051807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:55.051807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:55.051807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:55.051807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:55.051807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:55.051807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:55.051807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:00:55.052669 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:00:55.052669 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:55.052669 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:55.052669 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 450/896] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:00:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 16:00:55.058411 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:55.063743 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d787f58f-c519-4d17-99b0-0bfe984d3712 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:55.089788 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e870a6fa-cf56-487d-89ca-9d83762de43e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:55.091062 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e870a6fa-cf56-487d-89ca-9d83762de43e None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:55.091342 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e870a6fa-cf56-487d-89ca-9d83762de43e None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:55.091617 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e870a6fa-cf56-487d-89ca-9d83762de43e None admin] PATH_INFO: `/v3/services` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:55.092413 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e870a6fa-cf56-487d-89ca-9d83762de43e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:55.092777 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e870a6fa-cf56-487d-89ca-9d83762de43e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:55.094618 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e870a6fa-cf56-487d-89ca-9d83762de43e None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:55.102503 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 447/897] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Sep 19 16:00:54 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:00:55.106941 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e870a6fa-cf56-487d-89ca-9d83762de43e None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:55.210405 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d787f58f-c519-4d17-99b0-0bfe984d3712 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:55.217493 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e870a6fa-cf56-487d-89ca-9d83762de43e None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:55.253226 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e2c6a6-f4e3-4060-8b7a-64f53a875ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:55.253834 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e2c6a6-f4e3-4060-8b7a-64f53a875ee4 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:55.254109 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e2c6a6-f4e3-4060-8b7a-64f53a875ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:55.254396 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e2c6a6-f4e3-4060-8b7a-64f53a875ee4 None admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:55.255058 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e2c6a6-f4e3-4060-8b7a-64f53a875ee4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:55.255357 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e2c6a6-f4e3-4060-8b7a-64f53a875ee4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:55.255663 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a138713-7696-46d0-9e9c-77160502bf5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:55.256275 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a138713-7696-46d0-9e9c-77160502bf5a None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:55.256556 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a138713-7696-46d0-9e9c-77160502bf5a None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:55.256736 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e2c6a6-f4e3-4060-8b7a-64f53a875ee4 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:55.256845 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a138713-7696-46d0-9e9c-77160502bf5a None admin] PATH_INFO: `/v3/services` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:55.257445 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a138713-7696-46d0-9e9c-77160502bf5a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:55.257751 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a138713-7696-46d0-9e9c-77160502bf5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:55.259780 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:55.261183 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a138713-7696-46d0-9e9c-77160502bf5a None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:55.273737 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 451/899] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:00:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:55.274003 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 448/899] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Sep 19 16:00:55 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:00:55.274758 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e2c6a6-f4e3-4060-8b7a-64f53a875ee4 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:55.276462 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a138713-7696-46d0-9e9c-77160502bf5a None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:55.321403 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:55.399323 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e2c6a6-f4e3-4060-8b7a-64f53a875ee4 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:55.414423 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a138713-7696-46d0-9e9c-77160502bf5a None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:55.452062 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe73919-4aa9-4c33-90a7-0007dea2adab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:55.452697 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe73919-4aa9-4c33-90a7-0007dea2adab None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:55.452970 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe73919-4aa9-4c33-90a7-0007dea2adab None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:55.453238 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe73919-4aa9-4c33-90a7-0007dea2adab None admin] PATH_INFO: `/v3/endpoints` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:55.453969 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe73919-4aa9-4c33-90a7-0007dea2adab None admin] RBAC: Adding query filter params (interface=public, service_id=d8d823b8dd06404993e47ff0a2f6ce00, region_id=RegionOne) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:55.454273 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe73919-4aa9-4c33-90a7-0007dea2adab None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:55.455661 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe73919-4aa9-4c33-90a7-0007dea2adab None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:55.456070 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7e3de6-114c-4ba9-b73b-39e1a3ed658e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:55.456777 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e3de6-114c-4ba9-b73b-39e1a3ed658e None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:55.457375 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e3de6-114c-4ba9-b73b-39e1a3ed658e None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:55.457663 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e3de6-114c-4ba9-b73b-39e1a3ed658e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:55.462262 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 452/900] 10.222.0.57 () {56 vars in 1251 bytes} [Mon Sep 19 16:00:55 2022] GET /identity/v3/endpoints?service_id=d8d823b8dd06404993e47ff0a2f6ce00&interface=public®ion_id=RegionOne => generated 194 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:55.467410 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe73919-4aa9-4c33-90a7-0007dea2adab None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:55.471415 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7e3de6-114c-4ba9-b73b-39e1a3ed658e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:55.472946 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7e3de6-114c-4ba9-b73b-39e1a3ed658e None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:55.485669 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-5b7e3de6-114c-4ba9-b73b-39e1a3ed658e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 16:00:55.485669 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:55.485669 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:55.485669 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:55.485669 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:55.485669 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:55.485669 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:55.485669 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:55.485669 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:55.485669 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:55.485669 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:55.485669 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:55.485669 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 16:00:55.485669 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 16:00:55.485669 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 16:00:55.485669 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 16:00:55.486878 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:55.486878 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:55.486878 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 16:00:55.486878 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:00:55.486878 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 16:00:55.486878 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:00:55.486878 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:55.486878 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:55.486878 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:55.486878 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:55.486878 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:55.486878 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:55.486878 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:55.486878 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:55.486878 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:55.486878 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:55.486878 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:55.486878 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:55.488875 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:55.488875 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:55.488875 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 16:00:55.488875 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 16:00:55.488875 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 16:00:55.488875 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 16:00:55.488875 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 16:00:55.488875 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 16:00:55.488875 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 16:00:55.488875 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:55.488875 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 449/901] 10.222.0.57 () {56 vars in 1117 bytes} [Mon Sep 19 16:00:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 16:00:55.500261 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7e3de6-114c-4ba9-b73b-39e1a3ed658e None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:55.536122 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:55.585287 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe73919-4aa9-4c33-90a7-0007dea2adab None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:55.638748 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:55.649080 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7e3de6-114c-4ba9-b73b-39e1a3ed658e None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:55.649756 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be958e40-b289-4812-b6ec-647ad427736d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:55.650930 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be958e40-b289-4812-b6ec-647ad427736d None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:55.651395 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be958e40-b289-4812-b6ec-647ad427736d None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:55.652944 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be958e40-b289-4812-b6ec-647ad427736d None admin] PATH_INFO: `/v3/services` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:55.653766 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be958e40-b289-4812-b6ec-647ad427736d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:55.655348 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be958e40-b289-4812-b6ec-647ad427736d None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:55.664686 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 453/902] 10.222.0.57 () {56 vars in 1087 bytes} [Mon Sep 19 16:00:55 2022] GET /identity/v3/services => generated 1268 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 16:00:55.696197 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e201b0e-d68c-463d-9f14-ef7e783d5fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:55.697113 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e201b0e-d68c-463d-9f14-ef7e783d5fb6 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:55.697550 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e201b0e-d68c-463d-9f14-ef7e783d5fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:55.697974 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e201b0e-d68c-463d-9f14-ef7e783d5fb6 None admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:55.698940 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e201b0e-d68c-463d-9f14-ef7e783d5fb6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:55.699551 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e201b0e-d68c-463d-9f14-ef7e783d5fb6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:55.701296 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e201b0e-d68c-463d-9f14-ef7e783d5fb6 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:55.724176 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 450/903] 10.222.0.57 () {56 vars in 1149 bytes} [Mon Sep 19 16:00:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:55.731806 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be958e40-b289-4812-b6ec-647ad427736d None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:55.872973 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be958e40-b289-4812-b6ec-647ad427736d None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:55.898727 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:55.911467 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfb01f3-7f19-4801-8eed-e2d4f07af08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:55.912139 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb01f3-7f19-4801-8eed-e2d4f07af08d None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:55.912430 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb01f3-7f19-4801-8eed-e2d4f07af08d None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:55.912708 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb01f3-7f19-4801-8eed-e2d4f07af08d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:55.993340 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfb01f3-7f19-4801-8eed-e2d4f07af08d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:55.994894 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfb01f3-7f19-4801-8eed-e2d4f07af08d None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:56.104509 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 454/904] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 19 16:00:55 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 16:00:56.111732 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e201b0e-d68c-463d-9f14-ef7e783d5fb6 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:56.222400 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e201b0e-d68c-463d-9f14-ef7e783d5fb6 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:56.260954 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be653919-c7c1-4282-93ba-dcdf1ac6b94e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:56.261768 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be653919-c7c1-4282-93ba-dcdf1ac6b94e None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:56.262593 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be653919-c7c1-4282-93ba-dcdf1ac6b94e None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:56.262871 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be653919-c7c1-4282-93ba-dcdf1ac6b94e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:56.275039 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be653919-c7c1-4282-93ba-dcdf1ac6b94e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:56.276525 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be653919-c7c1-4282-93ba-dcdf1ac6b94e None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:56.295604 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-be653919-c7c1-4282-93ba-dcdf1ac6b94e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:56.295604 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:56.295604 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:56.295604 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:56.295604 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:56.295604 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:56.295604 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:56.295604 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:56.295604 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:56.295604 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:56.295604 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:56.295604 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:56.295604 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:00:56.295604 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:00:56.295604 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:00:56.295604 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:00:56.298071 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:56.298071 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:56.298071 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:56.298071 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:56.298071 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:56.298071 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:56.298071 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:56.298071 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:56.298071 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:56.298071 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:56.298071 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:56.298071 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:56.298071 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:56.298071 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:56.298071 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:56.298071 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:56.298071 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:00:56.300021 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:00:56.300021 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:00:56.300021 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:00:56.300021 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 451/905] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:00:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:56.303160 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfb01f3-7f19-4801-8eed-e2d4f07af08d None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:56.329185 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:56.381427 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c45b6f5-e8f0-46ef-aa8b-520301d559b1 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:56.381741 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c45b6f5-e8f0-46ef-aa8b-520301d559b1 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:56.381960 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c45b6f5-e8f0-46ef-aa8b-520301d559b1 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:56.384731 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 452/906] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:00:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 16:00:56.389080 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4e037e-8462-48f0-93d1-ff9d56c30ae2 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:56.389359 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4e037e-8462-48f0-93d1-ff9d56c30ae2 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:56.389653 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4e037e-8462-48f0-93d1-ff9d56c30ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:56.396395 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:56.411848 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-1f4e037e-8462-48f0-93d1-ff9d56c30ae2 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:00:56.413041 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-1f4e037e-8462-48f0-93d1-ff9d56c30ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:00:56.413391 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfb01f3-7f19-4801-8eed-e2d4f07af08d None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:56.460560 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97550d0a-f631-4180-8174-ece16a122d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:56.461180 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97550d0a-f631-4180-8174-ece16a122d61 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:56.461459 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97550d0a-f631-4180-8174-ece16a122d61 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:56.461734 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97550d0a-f631-4180-8174-ece16a122d61 None admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:56.462904 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97550d0a-f631-4180-8174-ece16a122d61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:56.463213 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97550d0a-f631-4180-8174-ece16a122d61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:56.465965 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97550d0a-f631-4180-8174-ece16a122d61 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:56.483541 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 455/907] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:00:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:00:56.488951 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97550d0a-f631-4180-8174-ece16a122d61 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:56.495571 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 453/908] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:00:56 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 16:00:56.584121 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:56.603348 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97550d0a-f631-4180-8174-ece16a122d61 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:56.642776 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d9a872-2a19-4359-910a-9b346f6262a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:56.643463 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d9a872-2a19-4359-910a-9b346f6262a2 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:56.643743 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d9a872-2a19-4359-910a-9b346f6262a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:56.644060 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d9a872-2a19-4359-910a-9b346f6262a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:56.651318 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d9a872-2a19-4359-910a-9b346f6262a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:56.653466 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d9a872-2a19-4359-910a-9b346f6262a2 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:56.666217 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-34d9a872-2a19-4359-910a-9b346f6262a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:00:56.666217 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:56.666217 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:56.666217 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:56.666217 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:56.666217 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:56.666217 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:56.666217 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:56.666217 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:56.666217 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:56.666217 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:56.666217 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:56.666217 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 16:00:56.666217 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 16:00:56.666217 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 16:00:56.668397 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 16:00:56.668397 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:56.668397 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:56.668397 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:56.668397 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:56.668397 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:56.668397 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:56.668397 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:56.668397 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:56.668397 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:56.668397 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:56.668397 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:56.668397 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:56.668397 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:56.668397 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:56.668397 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:56.668397 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:56.669956 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 16:00:56.669956 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 16:00:56.669956 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 16:00:56.669956 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 16:00:56.669956 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 16:00:56.669956 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 16:00:56.669956 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:00:56.669956 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:56.669956 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 456/909] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 19 16:00:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 16:00:56.673520 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4e037e-8462-48f0-93d1-ff9d56c30ae2 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:56.770475 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d9a872-2a19-4359-910a-9b346f6262a2 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:56.779065 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4e037e-8462-48f0-93d1-ff9d56c30ae2 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:56.811907 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:56.816342 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0450337b-9ca8-4035-969d-262e2134d2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:56.817119 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0450337b-9ca8-4035-969d-262e2134d2f4 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:56.817403 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0450337b-9ca8-4035-969d-262e2134d2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:56.817766 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0450337b-9ca8-4035-969d-262e2134d2f4 None admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:56.818852 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0450337b-9ca8-4035-969d-262e2134d2f4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:56.819220 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0450337b-9ca8-4035-969d-262e2134d2f4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:56.820989 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0450337b-9ca8-4035-969d-262e2134d2f4 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:56.838203 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 454/910] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 19 16:00:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:56.846502 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0450337b-9ca8-4035-969d-262e2134d2f4 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:56.904610 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d9a872-2a19-4359-910a-9b346f6262a2 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:56.959789 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df74f37-a9be-47e5-8236-98b51784d16a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:56.960455 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df74f37-a9be-47e5-8236-98b51784d16a None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:56.960734 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df74f37-a9be-47e5-8236-98b51784d16a None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:56.961010 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df74f37-a9be-47e5-8236-98b51784d16a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:56.961913 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df74f37-a9be-47e5-8236-98b51784d16a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:56.964028 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df74f37-a9be-47e5-8236-98b51784d16a None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:56.975870 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-9df74f37-a9be-47e5-8236-98b51784d16a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 19 16:00:56.975870 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:56.975870 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 16:00:56.975870 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 16:00:56.975870 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 16:00:56.975870 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 16:00:56.975870 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 16:00:56.975870 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 16:00:56.975870 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 19 16:00:56.975870 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:56.975870 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 16:00:56.975870 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:56.975870 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:56.975870 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:56.975870 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:56.975870 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:56.977284 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:56.977284 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:56.977284 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:56.977284 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:56.977284 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:56.977284 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:56.977284 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:56.977284 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 16:00:56.977284 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 16:00:56.977284 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 16:00:56.977284 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 16:00:56.977284 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:56.977284 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:56.977284 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:56.977284 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:56.977284 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:56.977284 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:56.978151 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:56.978151 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:56.978151 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:56.978151 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:56.978151 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:56.978151 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:56.978151 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:56.978151 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:56.978151 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:56.978151 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:56.978151 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 16:00:56.978151 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 16:00:56.978151 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 19 16:00:56.978151 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:56.978151 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 457/911] 10.222.0.57 () {56 vars in 1132 bytes} [Mon Sep 19 16:00:56 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 16:00:56.983011 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df74f37-a9be-47e5-8236-98b51784d16a None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:56.990101 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0450337b-9ca8-4035-969d-262e2134d2f4 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:56.998929 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:57.033742 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb07430-2029-4c1f-a74c-b7f281cdfcd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:57.034433 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb07430-2029-4c1f-a74c-b7f281cdfcd9 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:57.034698 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb07430-2029-4c1f-a74c-b7f281cdfcd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:57.034974 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb07430-2029-4c1f-a74c-b7f281cdfcd9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:57.035741 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb07430-2029-4c1f-a74c-b7f281cdfcd9 None admin] RBAC: Adding query filter params (role.id=c30e151440d44fd18e607056b4d41ec7, scope.project.id=be03e5300b3343169f35c643c93d1e06, user.id=7222bd3b313b47e5a4fa0ebc9ff0cdd5) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:57.036054 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb07430-2029-4c1f-a74c-b7f281cdfcd9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:57.037574 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb07430-2029-4c1f-a74c-b7f281cdfcd9 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:57.044682 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 455/912] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 19 16:00:56 2022] GET /identity/v3/role_assignments?user.id=7222bd3b313b47e5a4fa0ebc9ff0cdd5&scope.project.id=be03e5300b3343169f35c643c93d1e06&role.id=c30e151440d44fd18e607056b4d41ec7 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:57.045132 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836d0dce-cd6d-4939-b9d3-95f8644f23bc None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:57.045375 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836d0dce-cd6d-4939-b9d3-95f8644f23bc None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:57.045652 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836d0dce-cd6d-4939-b9d3-95f8644f23bc None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:57.047146 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 456/913] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:00:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:00:57.053081 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4e7ae-5990-4257-8884-d601068a2ed8 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:57.053462 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4e7ae-5990-4257-8884-d601068a2ed8 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:57.053805 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4e7ae-5990-4257-8884-d601068a2ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:57.075391 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-dae4e7ae-5990-4257-8884-d601068a2ed8 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:00:57.076858 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-dae4e7ae-5990-4257-8884-d601068a2ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:00:57.132275 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df74f37-a9be-47e5-8236-98b51784d16a None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:57.154083 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:57.175081 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5debf8-67c6-4d09-8bfb-d35fed1621df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:57.175809 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5debf8-67c6-4d09-8bfb-d35fed1621df None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:57.176164 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5debf8-67c6-4d09-8bfb-d35fed1621df None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:57.176509 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5debf8-67c6-4d09-8bfb-d35fed1621df None admin] PATH_INFO: `/v3/services` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:57.177224 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5debf8-67c6-4d09-8bfb-d35fed1621df None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:57.177663 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5debf8-67c6-4d09-8bfb-d35fed1621df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:57.179255 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5debf8-67c6-4d09-8bfb-d35fed1621df None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:57.186426 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 457/914] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:00:57 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 16:00:57.191791 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 458/915] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Sep 19 16:00:56 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:57.197792 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae4e7ae-5990-4257-8884-d601068a2ed8 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:57.313485 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae4e7ae-5990-4257-8884-d601068a2ed8 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:57.348842 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad23f64c-08d5-476f-b91d-2251e7a0ddec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:57.349699 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad23f64c-08d5-476f-b91d-2251e7a0ddec None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:57.350155 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad23f64c-08d5-476f-b91d-2251e7a0ddec None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:57.350614 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad23f64c-08d5-476f-b91d-2251e7a0ddec None admin] PATH_INFO: `/v3/services` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:57.351388 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad23f64c-08d5-476f-b91d-2251e7a0ddec None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:57.351842 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad23f64c-08d5-476f-b91d-2251e7a0ddec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:57.353409 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad23f64c-08d5-476f-b91d-2251e7a0ddec None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:57.359502 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 458/916] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Sep 19 16:00:57 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:57.365300 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5debf8-67c6-4d09-8bfb-d35fed1621df None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:57.442769 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:57.468947 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad23f64c-08d5-476f-b91d-2251e7a0ddec None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:57.473852 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5debf8-67c6-4d09-8bfb-d35fed1621df None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:57.511868 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2ed471-3c71-4ecc-b5b2-e0e20183da5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:57.512874 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2ed471-3c71-4ecc-b5b2-e0e20183da5d None admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:57.513339 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2ed471-3c71-4ecc-b5b2-e0e20183da5d None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:57.513805 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2ed471-3c71-4ecc-b5b2-e0e20183da5d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:57.514824 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2ed471-3c71-4ecc-b5b2-e0e20183da5d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:57.516521 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2ed471-3c71-4ecc-b5b2-e0e20183da5d None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:57.523148 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:57.546432 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 459/917] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:00:57 2022] POST /identity/v3/endpoints => generated 347 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:00:57.591913 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad23f64c-08d5-476f-b91d-2251e7a0ddec None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:57.644762 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237b1535-cce1-4847-af1e-9b9d4239ba94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:57.645769 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b1535-cce1-4847-af1e-9b9d4239ba94 None admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:57.646269 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b1535-cce1-4847-af1e-9b9d4239ba94 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:57.646805 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b1535-cce1-4847-af1e-9b9d4239ba94 None admin] PATH_INFO: `/v3/services` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:57.647706 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237b1535-cce1-4847-af1e-9b9d4239ba94 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:57.649430 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237b1535-cce1-4847-af1e-9b9d4239ba94 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:57.671266 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 459/918] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Sep 19 16:00:57 2022] POST /identity/v3/services => generated 241 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 16:00:58.051581 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ff8e4-093b-4199-9b34-4d791ef3af6e None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:58.052106 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ff8e4-093b-4199-9b34-4d791ef3af6e None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:58.052106 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ff8e4-093b-4199-9b34-4d791ef3af6e None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:58.053703 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 460/919] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:00:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:00:58.060039 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7209880-a830-4ba1-ae3c-c9cb79b29ff8 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:58.060481 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7209880-a830-4ba1-ae3c-c9cb79b29ff8 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:58.060661 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7209880-a830-4ba1-ae3c-c9cb79b29ff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:58.079234 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-a7209880-a830-4ba1-ae3c-c9cb79b29ff8 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:00:58.080452 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-a7209880-a830-4ba1-ae3c-c9cb79b29ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:00:58.159006 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 460/920] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:00:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 16:00:58.434675 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24ff8e4-093b-4199-9b34-4d791ef3af6e None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:58.547033 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24ff8e4-093b-4199-9b34-4d791ef3af6e None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:58.574880 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:58.583285 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043ba10a-8674-43c8-a4ce-c6f850a70799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:58.583917 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043ba10a-8674-43c8-a4ce-c6f850a70799 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:58.584199 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043ba10a-8674-43c8-a4ce-c6f850a70799 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:58.584468 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043ba10a-8674-43c8-a4ce-c6f850a70799 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:58.585115 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043ba10a-8674-43c8-a4ce-c6f850a70799 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:58.586574 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043ba10a-8674-43c8-a4ce-c6f850a70799 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:58.597587 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-043ba10a-8674-43c8-a4ce-c6f850a70799 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 19 16:00:58.597587 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:58.597587 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 16:00:58.597587 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 16:00:58.597587 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 16:00:58.597587 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 16:00:58.597587 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 16:00:58.597587 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 16:00:58.597587 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 19 16:00:58.597587 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:58.597587 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 16:00:58.597587 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:58.597587 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:58.597587 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:58.597587 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:58.597587 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:58.597587 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:58.598621 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:58.598621 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:58.598621 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:58.598621 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:58.598621 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:58.598621 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:58.598621 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 16:00:58.598621 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 16:00:58.598621 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 16:00:58.598621 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 16:00:58.598621 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:58.598621 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:58.598621 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:58.598621 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:58.598621 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:58.598621 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:58.599958 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:58.599958 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:58.599958 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:58.599958 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:58.599958 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:58.599958 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:58.599958 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:58.599958 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:58.599958 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:58.599958 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:58.599958 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 16:00:58.599958 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 16:00:58.599958 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 19 16:00:58.599958 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:58.599958 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 461/921] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Sep 19 16:00:58 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:58.606797 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7209880-a830-4ba1-ae3c-c9cb79b29ff8 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:58.650898 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:58.733747 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7209880-a830-4ba1-ae3c-c9cb79b29ff8 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:58.772520 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07c3bdc-4536-4f72-82c7-f49f1d3a4d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:58.773145 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07c3bdc-4536-4f72-82c7-f49f1d3a4d2d None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:58.773706 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07c3bdc-4536-4f72-82c7-f49f1d3a4d2d None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:58.774038 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07c3bdc-4536-4f72-82c7-f49f1d3a4d2d None admin] PATH_INFO: `/v3/services` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:58.774711 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07c3bdc-4536-4f72-82c7-f49f1d3a4d2d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:58.775021 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07c3bdc-4536-4f72-82c7-f49f1d3a4d2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:58.776448 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07c3bdc-4536-4f72-82c7-f49f1d3a4d2d None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:58.788645 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 461/922] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Sep 19 16:00:58 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:58.794599 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043ba10a-8674-43c8-a4ce-c6f850a70799 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:58.901402 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043ba10a-8674-43c8-a4ce-c6f850a70799 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:58.937854 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2faeff2-9089-4b8f-a39b-1675c53acbf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:58.938802 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2faeff2-9089-4b8f-a39b-1675c53acbf0 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:58.939290 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2faeff2-9089-4b8f-a39b-1675c53acbf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:58.939743 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2faeff2-9089-4b8f-a39b-1675c53acbf0 None admin] PATH_INFO: `/v3/services` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:58.940549 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2faeff2-9089-4b8f-a39b-1675c53acbf0 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:00:58.941031 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2faeff2-9089-4b8f-a39b-1675c53acbf0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:58.942580 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2faeff2-9089-4b8f-a39b-1675c53acbf0 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:58.948426 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 462/923] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Sep 19 16:00:58 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:00:58.958472 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07c3bdc-4536-4f72-82c7-f49f1d3a4d2d None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:59.061671 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:59.069496 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07c3bdc-4536-4f72-82c7-f49f1d3a4d2d None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:59.106037 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c88346e-998b-40c7-a111-c1281ff19ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:59.106680 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c88346e-998b-40c7-a111-c1281ff19ea6 None admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:59.106962 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c88346e-998b-40c7-a111-c1281ff19ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:59.107236 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c88346e-998b-40c7-a111-c1281ff19ea6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:59.108017 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c88346e-998b-40c7-a111-c1281ff19ea6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:00:59.109452 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c88346e-998b-40c7-a111-c1281ff19ea6 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:00:59.132900 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 462/924] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:00:58 2022] POST /identity/v3/endpoints => generated 346 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:00:59.384872 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9099ddd2-c655-49a6-86cd-6f55e45dca57 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:59.385352 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9099ddd2-c655-49a6-86cd-6f55e45dca57 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:59.385352 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9099ddd2-c655-49a6-86cd-6f55e45dca57 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:59.387005 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 463/925] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:00:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:00:59.392329 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e739b-593e-468c-ac90-83bba614db87 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:59.392606 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e739b-593e-468c-ac90-83bba614db87 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:59.392886 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e739b-593e-468c-ac90-83bba614db87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:59.412278 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-669e739b-593e-468c-ac90-83bba614db87 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:00:59.413476 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-669e739b-593e-468c-ac90-83bba614db87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:00:59.491129 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 463/926] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:00:59 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 16:00:59.769330 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9099ddd2-c655-49a6-86cd-6f55e45dca57 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:59.840952 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:00:59.891362 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9099ddd2-c655-49a6-86cd-6f55e45dca57 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:00:59.927827 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a9138e-0abc-4cc5-836c-6c89ff1d549f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:00:59.928461 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a9138e-0abc-4cc5-836c-6c89ff1d549f None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:59.928803 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a9138e-0abc-4cc5-836c-6c89ff1d549f None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:59.929074 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a9138e-0abc-4cc5-836c-6c89ff1d549f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:59.940382 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a9138e-0abc-4cc5-836c-6c89ff1d549f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 16:00:59.944119 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0138c8-ab8a-4994-bf3d-6f813ad007dc None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:59.944733 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0138c8-ab8a-4994-bf3d-6f813ad007dc None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:59.945231 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0138c8-ab8a-4994-bf3d-6f813ad007dc None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:59.945415 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-41a9138e-0abc-4cc5-836c-6c89ff1d549f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 16:00:59.945415 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:00:59.945415 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:00:59.945415 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:00:59.945415 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:00:59.945415 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:00:59.945415 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:00:59.945415 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:00:59.945415 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:00:59.945415 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:00:59.945415 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:00:59.945415 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:00:59.945415 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 16:00:59.945415 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 16:00:59.945415 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 16:00:59.945415 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 16:00:59.946289 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 16:00:59.946289 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 16:00:59.946289 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 16:00:59.946289 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 16:00:59.946289 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:00:59.946289 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:00:59.946289 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:00:59.946289 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:00:59.946289 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:00:59.946289 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:00:59.946289 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:00:59.946289 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:00:59.946289 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:00:59.946289 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:00:59.946289 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:00:59.946289 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:00:59.947571 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:00:59.947571 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:00:59.947571 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:00:59.947571 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:00:59.947571 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 16:00:59.947571 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 16:00:59.947571 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 16:00:59.947571 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 16:00:59.947571 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 16:00:59.947571 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 16:00:59.947571 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 16:00:59.947571 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:00:59.947571 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 464/927] 10.222.0.57 () {56 vars in 1120 bytes} [Mon Sep 19 16:00:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:00:59.950260 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 464/928] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:00:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 16:00:59.954685 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a9138e-0abc-4cc5-836c-6c89ff1d549f None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:00:59.956766 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b2acf-ebe2-45a8-a546-bb8c49fbb7f4 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:00:59.957340 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b2acf-ebe2-45a8-a546-bb8c49fbb7f4 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:00:59.957820 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b2acf-ebe2-45a8-a546-bb8c49fbb7f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:00:59.981727 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-a55b2acf-ebe2-45a8-a546-bb8c49fbb7f4 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:00:59.983986 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-a55b2acf-ebe2-45a8-a546-bb8c49fbb7f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:01:00.084654 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a9138e-0abc-4cc5-836c-6c89ff1d549f None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:00.087103 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 465/929] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:00:59 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 16:01:00.088867 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf010073-5b09-4ff6-b7b8-e5dc980007bb None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:00.089149 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf010073-5b09-4ff6-b7b8-e5dc980007bb None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:00.089420 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf010073-5b09-4ff6-b7b8-e5dc980007bb None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:00.090698 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 466/930] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:01:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:01:00.096509 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64abd84-a806-48a2-a298-a185569d20ff None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:00.096786 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64abd84-a806-48a2-a298-a185569d20ff None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:00.097225 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64abd84-a806-48a2-a298-a185569d20ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:00.107212 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:00.119497 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-a64abd84-a806-48a2-a298-a185569d20ff None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:01:00.121373 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-a64abd84-a806-48a2-a298-a185569d20ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:01:00.124017 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:00.132042 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c9c3be-33a9-4704-a58e-f9deb1c3582e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:00.132962 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c9c3be-33a9-4704-a58e-f9deb1c3582e None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:00.133456 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c9c3be-33a9-4704-a58e-f9deb1c3582e None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:00.133729 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c9c3be-33a9-4704-a58e-f9deb1c3582e None admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:00.134442 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c9c3be-33a9-4704-a58e-f9deb1c3582e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:00.134759 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c9c3be-33a9-4704-a58e-f9deb1c3582e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:00.136164 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c9c3be-33a9-4704-a58e-f9deb1c3582e None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:00.148556 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 465/931] 10.222.0.57 () {56 vars in 1115 bytes} [Mon Sep 19 16:00:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:01:00.153726 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c9c3be-33a9-4704-a58e-f9deb1c3582e None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:00.206605 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 467/932] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:01:00 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 16:01:00.269990 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c9c3be-33a9-4704-a58e-f9deb1c3582e None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:00.306294 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d0b20f-2264-47de-831f-eb3777cb37d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:00.306947 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d0b20f-2264-47de-831f-eb3777cb37d9 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:00.307223 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d0b20f-2264-47de-831f-eb3777cb37d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:00.307491 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d0b20f-2264-47de-831f-eb3777cb37d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:00.318633 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d0b20f-2264-47de-831f-eb3777cb37d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:00.320105 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d0b20f-2264-47de-831f-eb3777cb37d9 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:00.335354 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-51d0b20f-2264-47de-831f-eb3777cb37d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:01:00.335354 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:01:00.335354 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:01:00.335354 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:01:00.335354 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:01:00.335354 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:01:00.335354 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:01:00.335354 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:01:00.335354 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:01:00.335354 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:01:00.335354 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:01:00.335354 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:01:00.335354 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:01:00.335354 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:01:00.335354 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:01:00.335354 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:01:00.336263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:01:00.336263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:01:00.336263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:01:00.336263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:01:00.336263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:01:00.336263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:01:00.336263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:01:00.336263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:01:00.336263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:01:00.336263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:01:00.336263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:01:00.336263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:01:00.336263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:01:00.336263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:01:00.336263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:01:00.336263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:01:00.336263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:01:00.337555 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:01:00.337555 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:01:00.337555 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:01:00.337555 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 466/933] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:01:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 16:01:00.343158 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64abd84-a806-48a2-a298-a185569d20ff None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:00.375903 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d0b20f-2264-47de-831f-eb3777cb37d9 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:00.419085 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:00.471287 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64abd84-a806-48a2-a298-a185569d20ff None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:00.495112 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d0b20f-2264-47de-831f-eb3777cb37d9 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:00.520419 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe97e7f-77a9-40a4-a3e3-d6aea3c70e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:00.521214 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe97e7f-77a9-40a4-a3e3-d6aea3c70e76 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:00.521492 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe97e7f-77a9-40a4-a3e3-d6aea3c70e76 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:00.521760 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe97e7f-77a9-40a4-a3e3-d6aea3c70e76 None admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:00.522445 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe97e7f-77a9-40a4-a3e3-d6aea3c70e76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:00.522947 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe97e7f-77a9-40a4-a3e3-d6aea3c70e76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:00.524373 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe97e7f-77a9-40a4-a3e3-d6aea3c70e76 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:00.533563 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:00.536360 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc03c94e-21f8-4775-a10e-b83da5eb7cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:00.537090 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc03c94e-21f8-4775-a10e-b83da5eb7cad None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:00.537388 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc03c94e-21f8-4775-a10e-b83da5eb7cad None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:00.537653 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc03c94e-21f8-4775-a10e-b83da5eb7cad None admin] PATH_INFO: `/v3/services/compute` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:00.538208 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc03c94e-21f8-4775-a10e-b83da5eb7cad None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:00.539598 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc03c94e-21f8-4775-a10e-b83da5eb7cad None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:00.542532 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 468/934] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:01:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:00.543474 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe97e7f-77a9-40a4-a3e3-d6aea3c70e76 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:00.553990 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-cc03c94e-21f8-4775-a10e-b83da5eb7cad None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 19 16:01:00.553990 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:01:00.553990 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 16:01:00.553990 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 16:01:00.553990 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 16:01:00.553990 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 16:01:00.553990 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 16:01:00.553990 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 16:01:00.553990 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 19 16:01:00.553990 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:01:00.553990 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 16:01:00.553990 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:01:00.553990 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:01:00.553990 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:01:00.553990 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:01:00.553990 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:01:00.553990 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:01:00.555097 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:01:00.555097 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:01:00.555097 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:01:00.555097 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:01:00.555097 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:01:00.555097 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:01:00.555097 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 16:01:00.555097 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 16:01:00.555097 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 16:01:00.555097 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 16:01:00.555097 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:01:00.555097 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:01:00.555097 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:01:00.555097 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:01:00.555097 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:01:00.555097 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:01:00.555932 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:01:00.555932 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:01:00.555932 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:01:00.555932 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:01:00.555932 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:01:00.555932 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:01:00.555932 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:01:00.555932 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:01:00.555932 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:01:00.555932 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:01:00.555932 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 16:01:00.555932 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 16:01:00.555932 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 19 16:01:00.555932 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:01:00.557426 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 467/935] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 19 16:01:00 2022] GET /identity/v3/services/compute => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:01:00.557595 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc03c94e-21f8-4775-a10e-b83da5eb7cad None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:00.660509 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe97e7f-77a9-40a4-a3e3-d6aea3c70e76 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:00.670429 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:00.678431 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc03c94e-21f8-4775-a10e-b83da5eb7cad None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:00.697707 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6a203e-c7cd-452e-8cec-6fe2f421f9ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:00.698339 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a203e-c7cd-452e-8cec-6fe2f421f9ea None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:00.698655 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a203e-c7cd-452e-8cec-6fe2f421f9ea None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:00.698930 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a203e-c7cd-452e-8cec-6fe2f421f9ea None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:00.700314 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6a203e-c7cd-452e-8cec-6fe2f421f9ea None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:00.702448 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6a203e-c7cd-452e-8cec-6fe2f421f9ea None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:00.716564 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-cf6a203e-c7cd-452e-8cec-6fe2f421f9ea None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 16:01:00.716564 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:01:00.716564 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 16:01:00.716564 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 16:01:00.716564 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 16:01:00.716564 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 16:01:00.716564 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 16:01:00.716564 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 16:01:00.716564 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 16:01:00.716564 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:01:00.716564 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 16:01:00.716564 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:01:00.716564 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:01:00.716564 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:01:00.716564 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:01:00.716564 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:01:00.716564 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:01:00.717577 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:01:00.717577 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:01:00.717577 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:01:00.717577 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:01:00.717577 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:01:00.717577 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:01:00.717577 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 16:01:00.717577 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 16:01:00.717577 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 16:01:00.717577 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 16:01:00.717577 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:01:00.717577 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:01:00.717577 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:01:00.717577 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:01:00.717577 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:01:00.717577 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:01:00.718409 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:01:00.718409 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:01:00.718409 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:01:00.718409 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:01:00.718409 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:01:00.718409 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:01:00.718409 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:01:00.718409 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:01:00.718409 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:01:00.718409 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:01:00.718409 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 16:01:00.718409 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 16:01:00.718409 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 16:01:00.718409 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:01:00.719143 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 469/936] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Sep 19 16:01:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:01:00.720054 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6a203e-c7cd-452e-8cec-6fe2f421f9ea None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:00.736964 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf4e5cd-a31a-49cf-aceb-d17aa4fe788b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:00.738195 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf4e5cd-a31a-49cf-aceb-d17aa4fe788b None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:00.739216 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf4e5cd-a31a-49cf-aceb-d17aa4fe788b None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:00.739884 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf4e5cd-a31a-49cf-aceb-d17aa4fe788b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:00.752303 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf4e5cd-a31a-49cf-aceb-d17aa4fe788b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:00.754678 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf4e5cd-a31a-49cf-aceb-d17aa4fe788b None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:00.770327 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-acf4e5cd-a31a-49cf-aceb-d17aa4fe788b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 16:01:00.770327 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:01:00.770327 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:01:00.770327 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:01:00.770327 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:01:00.770327 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:01:00.770327 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:01:00.770327 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:01:00.770327 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:01:00.770327 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:01:00.770327 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:01:00.770327 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:01:00.770327 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 16:01:00.770327 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 16:01:00.770327 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 16:01:00.770327 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 16:01:00.771388 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:01:00.771388 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:01:00.771388 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 16:01:00.771388 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:01:00.771388 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 16:01:00.771388 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:01:00.771388 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:01:00.771388 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:01:00.771388 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:01:00.771388 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:01:00.771388 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:01:00.771388 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:01:00.771388 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:01:00.771388 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:01:00.771388 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:01:00.771388 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:01:00.771388 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:01:00.771388 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:01:00.772309 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:01:00.772309 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:01:00.772309 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 16:01:00.772309 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 16:01:00.772309 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 16:01:00.772309 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 16:01:00.772309 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 16:01:00.772309 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 16:01:00.772309 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 16:01:00.772309 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:01:00.772309 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 468/937] 10.222.0.57 () {56 vars in 1117 bytes} [Mon Sep 19 16:01:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:01:00.772894 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf4e5cd-a31a-49cf-aceb-d17aa4fe788b None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:00.804079 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:00.851841 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6a203e-c7cd-452e-8cec-6fe2f421f9ea None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:00.891649 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4781dfc-6cc3-4d34-9caf-e4ddad4d2ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:00.892304 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4781dfc-6cc3-4d34-9caf-e4ddad4d2ce1 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:00.892552 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4781dfc-6cc3-4d34-9caf-e4ddad4d2ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:00.892831 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4781dfc-6cc3-4d34-9caf-e4ddad4d2ce1 None admin] PATH_INFO: `/v3/services` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:00.893571 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4781dfc-6cc3-4d34-9caf-e4ddad4d2ce1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:00.893882 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4781dfc-6cc3-4d34-9caf-e4ddad4d2ce1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:00.895332 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4781dfc-6cc3-4d34-9caf-e4ddad4d2ce1 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:00.900764 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 470/938] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Sep 19 16:01:00 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:00.901609 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4781dfc-6cc3-4d34-9caf-e4ddad4d2ce1 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:00.912360 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf4e5cd-a31a-49cf-aceb-d17aa4fe788b None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:00.960878 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3f0841-96f6-44e4-98a6-1bb820cbbf17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:00.961975 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f0841-96f6-44e4-98a6-1bb820cbbf17 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:00.962250 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f0841-96f6-44e4-98a6-1bb820cbbf17 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:00.962568 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f0841-96f6-44e4-98a6-1bb820cbbf17 None admin] PATH_INFO: `/v3/services` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:00.963211 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3f0841-96f6-44e4-98a6-1bb820cbbf17 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:00.963517 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3f0841-96f6-44e4-98a6-1bb820cbbf17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:00.965808 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3f0841-96f6-44e4-98a6-1bb820cbbf17 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:00.985090 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 469/939] 10.222.0.57 () {56 vars in 1122 bytes} [Mon Sep 19 16:01:00 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:00.985090 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3f0841-96f6-44e4-98a6-1bb820cbbf17 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:01.006135 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:01.024574 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4781dfc-6cc3-4d34-9caf-e4ddad4d2ce1 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:01.052699 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:01.062929 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8310e483-4e6f-4690-bb9b-a7b0d690fd7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:01.063547 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8310e483-4e6f-4690-bb9b-a7b0d690fd7e None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:01.063829 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8310e483-4e6f-4690-bb9b-a7b0d690fd7e None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:01.064095 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8310e483-4e6f-4690-bb9b-a7b0d690fd7e None admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:01.064842 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8310e483-4e6f-4690-bb9b-a7b0d690fd7e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:01.065151 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8310e483-4e6f-4690-bb9b-a7b0d690fd7e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:01.066603 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8310e483-4e6f-4690-bb9b-a7b0d690fd7e None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:01.095546 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 471/940] 10.222.0.57 () {56 vars in 1149 bytes} [Mon Sep 19 16:01:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:01.095546 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8310e483-4e6f-4690-bb9b-a7b0d690fd7e None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:01.111507 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3f0841-96f6-44e4-98a6-1bb820cbbf17 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:01.148780 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3026019-836b-40bd-83f6-ca9bb13fa3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:01.149450 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3026019-836b-40bd-83f6-ca9bb13fa3c8 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:01.149732 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3026019-836b-40bd-83f6-ca9bb13fa3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:01.150013 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3026019-836b-40bd-83f6-ca9bb13fa3c8 None admin] PATH_INFO: `/v3/services` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:01.150670 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3026019-836b-40bd-83f6-ca9bb13fa3c8 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:01.150977 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3026019-836b-40bd-83f6-ca9bb13fa3c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:01.152360 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3026019-836b-40bd-83f6-ca9bb13fa3c8 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:01.166491 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 470/941] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Sep 19 16:01:00 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:01.166491 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3026019-836b-40bd-83f6-ca9bb13fa3c8 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:01.204394 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8310e483-4e6f-4690-bb9b-a7b0d690fd7e None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:01.248596 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad1da5-8bcb-4364-a791-48c1960ece96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:01.249225 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad1da5-8bcb-4364-a791-48c1960ece96 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:01.249504 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad1da5-8bcb-4364-a791-48c1960ece96 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:01.249777 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad1da5-8bcb-4364-a791-48c1960ece96 None admin] PATH_INFO: `/v3/services` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:01.250499 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ad1da5-8bcb-4364-a791-48c1960ece96 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:01.250763 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ad1da5-8bcb-4364-a791-48c1960ece96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:01.252270 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ad1da5-8bcb-4364-a791-48c1960ece96 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:01.258059 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 472/942] 10.222.0.57 () {56 vars in 1122 bytes} [Mon Sep 19 16:01:01 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:01.264368 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad1da5-8bcb-4364-a791-48c1960ece96 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:01.304675 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:01.324852 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:01.336761 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3026019-836b-40bd-83f6-ca9bb13fa3c8 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:01.391702 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad1da5-8bcb-4364-a791-48c1960ece96 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:01.392764 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b4487-7eab-4f2a-858b-789600479585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:01.393761 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b4487-7eab-4f2a-858b-789600479585 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:01.394756 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b4487-7eab-4f2a-858b-789600479585 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:01.395038 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b4487-7eab-4f2a-858b-789600479585 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:01.414156 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624b4487-7eab-4f2a-858b-789600479585 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:01.416966 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624b4487-7eab-4f2a-858b-789600479585 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:01.429065 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfccc15-274c-4ae9-89d2-6c7fb763e537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:01.429426 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfccc15-274c-4ae9-89d2-6c7fb763e537 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:01.429710 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfccc15-274c-4ae9-89d2-6c7fb763e537 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:01.429985 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfccc15-274c-4ae9-89d2-6c7fb763e537 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:01.430721 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfccc15-274c-4ae9-89d2-6c7fb763e537 None admin] RBAC: Adding query filter params (interface=public, service_id=ff34ff3102a44ff9bb52ee1d14a10487, region_id=RegionOne) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:01.431032 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfccc15-274c-4ae9-89d2-6c7fb763e537 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:01.432397 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfccc15-274c-4ae9-89d2-6c7fb763e537 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:01.442406 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 473/943] 10.222.0.57 () {56 vars in 1251 bytes} [Mon Sep 19 16:01:01 2022] GET /identity/v3/endpoints?service_id=ff34ff3102a44ff9bb52ee1d14a10487&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:01:01.442620 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-624b4487-7eab-4f2a-858b-789600479585 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:01:01.442620 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:01:01.442620 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:01:01.442620 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:01:01.442620 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:01:01.442620 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:01:01.442620 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:01:01.442620 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:01:01.442620 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:01:01.442620 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:01:01.442620 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:01:01.442620 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:01:01.442620 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:01:01.442620 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:01:01.442620 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:01:01.442620 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:01:01.443851 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:01:01.443851 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:01:01.443851 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:01:01.443851 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:01:01.443851 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:01:01.443851 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:01:01.443851 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:01:01.443851 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:01:01.443851 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:01:01.443851 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:01:01.443851 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:01:01.443851 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:01:01.443851 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:01:01.443851 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:01:01.443851 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:01:01.443851 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:01:01.443851 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:01:01.444749 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:01:01.444749 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:01:01.444749 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:01:01.446770 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 471/944] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:01:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:01:01.452451 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfccc15-274c-4ae9-89d2-6c7fb763e537 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:01.455281 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b4487-7eab-4f2a-858b-789600479585 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:01.561391 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfccc15-274c-4ae9-89d2-6c7fb763e537 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:01.570814 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b4487-7eab-4f2a-858b-789600479585 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:01.575309 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:01.598526 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f5aebd-1de7-4751-8646-6c0e5c5c5861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:01.599138 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f5aebd-1de7-4751-8646-6c0e5c5c5861 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:01.599419 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f5aebd-1de7-4751-8646-6c0e5c5c5861 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:01.599692 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f5aebd-1de7-4751-8646-6c0e5c5c5861 None admin] PATH_INFO: `/v3/services` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:01.600293 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f5aebd-1de7-4751-8646-6c0e5c5c5861 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:01.601661 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f5aebd-1de7-4751-8646-6c0e5c5c5861 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:01.607403 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40f1eb7-be9e-48fe-9733-b5ab9ed34f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:01.608015 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40f1eb7-be9e-48fe-9733-b5ab9ed34f8d None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:01.608292 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40f1eb7-be9e-48fe-9733-b5ab9ed34f8d None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:01.608563 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40f1eb7-be9e-48fe-9733-b5ab9ed34f8d None admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:01.609200 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40f1eb7-be9e-48fe-9733-b5ab9ed34f8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:01.609499 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40f1eb7-be9e-48fe-9733-b5ab9ed34f8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:01.610898 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40f1eb7-be9e-48fe-9733-b5ab9ed34f8d None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:01.612794 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:01.618829 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 474/945] 10.222.0.57 () {56 vars in 1087 bytes} [Mon Sep 19 16:01:01 2022] GET /identity/v3/services => generated 1497 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:01:01.622032 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b2bca-0a8c-46c5-88fb-815da19728a2 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:01.623993 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b2bca-0a8c-46c5-88fb-815da19728a2 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:01.624554 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b2bca-0a8c-46c5-88fb-815da19728a2 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:01.626839 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 475/946] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:01:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:01:01.631139 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 472/947] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:01:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:01:01.632605 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ef1d39-3d8f-4797-91ff-b6c3339fbf25 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:01.632890 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ef1d39-3d8f-4797-91ff-b6c3339fbf25 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:01.633162 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ef1d39-3d8f-4797-91ff-b6c3339fbf25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:01.640324 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40f1eb7-be9e-48fe-9733-b5ab9ed34f8d None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:01.658415 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-e6ef1d39-3d8f-4797-91ff-b6c3339fbf25 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:01:01.660027 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-e6ef1d39-3d8f-4797-91ff-b6c3339fbf25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:01:01.754801 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 476/948] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:01:01 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 16:01:01.771059 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40f1eb7-be9e-48fe-9733-b5ab9ed34f8d None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:01.810679 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009bfd22-0a82-4a12-8e7f-b16d42b41015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:01.811314 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009bfd22-0a82-4a12-8e7f-b16d42b41015 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:01.811594 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009bfd22-0a82-4a12-8e7f-b16d42b41015 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:01.811869 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009bfd22-0a82-4a12-8e7f-b16d42b41015 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:01.819643 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009bfd22-0a82-4a12-8e7f-b16d42b41015 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:01.821074 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009bfd22-0a82-4a12-8e7f-b16d42b41015 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:01.833256 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-009bfd22-0a82-4a12-8e7f-b16d42b41015 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:01:01.833256 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:01:01.833256 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:01:01.833256 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:01:01.833256 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:01:01.833256 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:01:01.833256 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:01:01.833256 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:01:01.833256 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:01:01.833256 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:01:01.833256 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:01:01.833256 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:01:01.833256 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 16:01:01.833256 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 16:01:01.833256 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 16:01:01.834174 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 16:01:01.834174 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:01:01.834174 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:01:01.834174 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:01:01.834174 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:01:01.834174 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:01:01.834174 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:01:01.834174 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:01:01.834174 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:01:01.834174 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:01:01.834174 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:01:01.834174 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:01:01.834174 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:01:01.834174 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:01:01.834174 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:01:01.834174 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:01:01.834174 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:01:01.835639 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 16:01:01.835639 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 16:01:01.835639 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 16:01:01.835639 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 16:01:01.835639 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 16:01:01.835639 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 16:01:01.835639 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:01:01.835639 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:01:01.835639 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 473/949] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 19 16:01:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 16:01:01.840779 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ef1d39-3d8f-4797-91ff-b6c3339fbf25 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:01.949570 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ef1d39-3d8f-4797-91ff-b6c3339fbf25 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:01.957659 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:01.989345 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40bcca1-e455-4b5f-9567-d78eae68ff1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:01.990015 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40bcca1-e455-4b5f-9567-d78eae68ff1d None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:01.990292 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40bcca1-e455-4b5f-9567-d78eae68ff1d None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:01.990589 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40bcca1-e455-4b5f-9567-d78eae68ff1d None admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:01.991339 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40bcca1-e455-4b5f-9567-d78eae68ff1d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:01.991644 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40bcca1-e455-4b5f-9567-d78eae68ff1d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:01.993084 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40bcca1-e455-4b5f-9567-d78eae68ff1d None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:02.006150 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 477/950] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 19 16:01:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:01:02.012149 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009bfd22-0a82-4a12-8e7f-b16d42b41015 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:02.037561 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40bcca1-e455-4b5f-9567-d78eae68ff1d None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:02.114763 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:02.135795 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009bfd22-0a82-4a12-8e7f-b16d42b41015 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:02.147499 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40bcca1-e455-4b5f-9567-d78eae68ff1d None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:02.175424 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afd51c5-926a-48b6-a0e0-05b6a379b5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:02.176183 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afd51c5-926a-48b6-a0e0-05b6a379b5c8 None admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:02.176468 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afd51c5-926a-48b6-a0e0-05b6a379b5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:02.176744 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afd51c5-926a-48b6-a0e0-05b6a379b5c8 None admin] PATH_INFO: `/v3/projects/be03e5300b3343169f35c643c93d1e06/users/7222bd3b313b47e5a4fa0ebc9ff0cdd5/roles/c30e151440d44fd18e607056b4d41ec7` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:02.179528 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afd51c5-926a-48b6-a0e0-05b6a379b5c8 None admin] RBAC: Authorizing `identity:create_grant(project_id=be03e5300b3343169f35c643c93d1e06, user_id=7222bd3b313b47e5a4fa0ebc9ff0cdd5, role_id=c30e151440d44fd18e607056b4d41ec7)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:02.181214 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afd51c5-926a-48b6-a0e0-05b6a379b5c8 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:02.184596 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3423f68d-6893-42ed-8a65-636d5095743e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:02.185348 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3423f68d-6893-42ed-8a65-636d5095743e None admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:02.185692 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3423f68d-6893-42ed-8a65-636d5095743e None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:02.185971 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3423f68d-6893-42ed-8a65-636d5095743e None admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:02.187623 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3423f68d-6893-42ed-8a65-636d5095743e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:02.189514 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3423f68d-6893-42ed-8a65-636d5095743e None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:02.192428 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: WARNING keystone.server.flask.common [None req-3423f68d-6893-42ed-8a65-636d5095743e None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 19 16:01:02.202647 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 474/951] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Sep 19 16:01:02 2022] PUT /identity/v3/projects/be03e5300b3343169f35c643c93d1e06/users/7222bd3b313b47e5a4fa0ebc9ff0cdd5/roles/c30e151440d44fd18e607056b4d41ec7 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:01:02.213913 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.sql.core [None req-3423f68d-6893-42ed-8a65-636d5095743e None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 19 16:01:02.213913 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 19 16:01:02.213913 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [parameters: {'id': 'e218bdfaa00345ebb59f14423a937e96', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 19 16:01:02.213913 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81617) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 19 16:01:02.233235 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-3423f68d-6893-42ed-8a65-636d5095743e None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e218bdfaa00345ebb59f14423a937e96.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e218bdfaa00345ebb59f14423a937e96. Sep 19 16:01:02.233235 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:01:02.233235 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 19 16:01:02.233235 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 19 16:01:02.233235 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 19 16:01:02.233235 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 19 16:01:02.233235 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 19 16:01:02.233235 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application result = self._query(query) Sep 19 16:01:02.233235 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 19 16:01:02.233235 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application conn.query(q) Sep 19 16:01:02.233235 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 19 16:01:02.233235 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 19 16:01:02.233235 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 19 16:01:02.234181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application result.read() Sep 19 16:01:02.234181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 19 16:01:02.234181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 19 16:01:02.234181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 19 16:01:02.234181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 19 16:01:02.234181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 19 16:01:02.234181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 19 16:01:02.234181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 19 16:01:02.234181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 19 16:01:02.234181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 19 16:01:02.234181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:01:02.234181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 19 16:01:02.234181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:01:02.234181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:01:02.234181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 19 16:01:02.234181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 19 16:01:02.234181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 19 16:01:02.234181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 19 16:01:02.235343 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 19 16:01:02.235343 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application next(self.gen) Sep 19 16:01:02.235343 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 19 16:01:02.235343 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application yield resource Sep 19 16:01:02.235343 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 19 16:01:02.235343 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application next(self.gen) Sep 19 16:01:02.235343 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 19 16:01:02.235343 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application self.session.rollback() Sep 19 16:01:02.235343 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 19 16:01:02.235343 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application self.force_reraise() Sep 19 16:01:02.235343 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 19 16:01:02.235343 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise self.value Sep 19 16:01:02.235343 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 19 16:01:02.235343 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 19 16:01:02.235343 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 19 16:01:02.235343 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application session.commit() Sep 19 16:01:02.235343 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 19 16:01:02.236449 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 19 16:01:02.236449 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 19 16:01:02.236449 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application self._prepare_impl() Sep 19 16:01:02.236449 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 19 16:01:02.236449 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application self.session.flush() Sep 19 16:01:02.236449 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 19 16:01:02.236449 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application self._flush(objects) Sep 19 16:01:02.236449 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 19 16:01:02.236449 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 19 16:01:02.236449 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 19 16:01:02.236449 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application compat.raise_( Sep 19 16:01:02.236449 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 19 16:01:02.236449 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception Sep 19 16:01:02.236449 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 19 16:01:02.236449 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application flush_context.execute() Sep 19 16:01:02.236449 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 19 16:01:02.236449 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rec.execute(self) Sep 19 16:01:02.237542 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 19 16:01:02.237542 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 19 16:01:02.237542 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 19 16:01:02.237542 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 19 16:01:02.237542 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 19 16:01:02.237542 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 19 16:01:02.237542 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 19 16:01:02.237542 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 19 16:01:02.237542 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 19 16:01:02.237542 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 19 16:01:02.237542 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 19 16:01:02.237542 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 19 16:01:02.237542 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 19 16:01:02.237542 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 19 16:01:02.237542 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 19 16:01:02.238701 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 19 16:01:02.238701 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 19 16:01:02.238701 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception Sep 19 16:01:02.238701 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 19 16:01:02.238701 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 19 16:01:02.238701 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 19 16:01:02.238701 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 19 16:01:02.238701 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 19 16:01:02.238701 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application result = self._query(query) Sep 19 16:01:02.238701 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 19 16:01:02.238701 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application conn.query(q) Sep 19 16:01:02.238701 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 19 16:01:02.238701 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 19 16:01:02.238701 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 19 16:01:02.238701 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application result.read() Sep 19 16:01:02.238701 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 19 16:01:02.238701 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 19 16:01:02.239872 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 19 16:01:02.239872 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 19 16:01:02.239872 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 19 16:01:02.239872 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 19 16:01:02.239872 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 19 16:01:02.239872 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 19 16:01:02.239872 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 19 16:01:02.239872 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 19 16:01:02.239872 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [parameters: {'id': 'e218bdfaa00345ebb59f14423a937e96', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 19 16:01:02.239872 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 19 16:01:02.239872 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:01:02.239872 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 16:01:02.239872 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:01:02.239872 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:01:02.239872 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 19 16:01:02.240918 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 19 16:01:02.240918 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 19 16:01:02.240918 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 19 16:01:02.240918 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 19 16:01:02.240918 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:01:02.240918 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 16:01:02.240918 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:01:02.240918 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:01:02.240918 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:01:02.240918 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:01:02.240918 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:01:02.240918 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:01:02.240918 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:01:02.240918 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:01:02.240918 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:01:02.240918 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:01:02.240918 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:01:02.243646 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:01:02.243646 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 19 16:01:02.243646 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 19 16:01:02.243646 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:01:02.243646 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:01:02.243646 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 19 16:01:02.243646 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 19 16:01:02.243646 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e218bdfaa00345ebb59f14423a937e96. Sep 19 16:01:02.243646 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:01:02.243646 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 478/952] 10.222.0.57 () {60 vars in 1140 bytes} [Mon Sep 19 16:01:02 2022] POST /identity/v3/projects => generated 262 bytes in 198 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 19 16:01:02.250828 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afd51c5-926a-48b6-a0e0-05b6a379b5c8 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:02.371664 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afd51c5-926a-48b6-a0e0-05b6a379b5c8 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:02.388018 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:02.413688 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8ed5fd-5b81-4d1a-840a-3b718ac362f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:02.414459 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ed5fd-5b81-4d1a-840a-3b718ac362f9 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:02.414771 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ed5fd-5b81-4d1a-840a-3b718ac362f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:02.415009 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ed5fd-5b81-4d1a-840a-3b718ac362f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:02.423486 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8ed5fd-5b81-4d1a-840a-3b718ac362f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:02.424881 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8ed5fd-5b81-4d1a-840a-3b718ac362f9 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:02.437501 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-5d8ed5fd-5b81-4d1a-840a-3b718ac362f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:01:02.437501 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:01:02.437501 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:01:02.437501 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:01:02.437501 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:01:02.437501 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:01:02.437501 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:01:02.437501 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:01:02.437501 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:01:02.437501 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:01:02.437501 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:01:02.437501 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:01:02.437501 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 16:01:02.437501 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 16:01:02.437501 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 16:01:02.438540 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 16:01:02.438540 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:01:02.438540 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:01:02.438540 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:01:02.438540 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:01:02.438540 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:01:02.438540 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:01:02.438540 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:01:02.438540 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:01:02.438540 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:01:02.438540 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:01:02.438540 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:01:02.438540 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:01:02.438540 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:01:02.438540 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:01:02.438540 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:01:02.438540 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:01:02.439522 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 16:01:02.439522 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 16:01:02.439522 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 16:01:02.439522 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 16:01:02.439522 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 16:01:02.439522 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 16:01:02.439522 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:01:02.439522 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:01:02.442440 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 475/953] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 19 16:01:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:01:02.445160 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3423f68d-6893-42ed-8a65-636d5095743e None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:02.475615 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:02.565241 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3423f68d-6893-42ed-8a65-636d5095743e None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:02.603628 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6230aef5-39cb-476d-8b2b-776b13045061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:02.604260 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230aef5-39cb-476d-8b2b-776b13045061 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:02.604826 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230aef5-39cb-476d-8b2b-776b13045061 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:02.605101 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230aef5-39cb-476d-8b2b-776b13045061 None admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:02.606330 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6230aef5-39cb-476d-8b2b-776b13045061 None admin] RBAC: Adding query filter params (name=service) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:02.606671 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6230aef5-39cb-476d-8b2b-776b13045061 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:02.608345 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6230aef5-39cb-476d-8b2b-776b13045061 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:02.631869 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 479/954] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Sep 19 16:01:02 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:03.426968 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d76f8-c628-4af2-b1da-86cde020cc14 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:03.428019 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d76f8-c628-4af2-b1da-86cde020cc14 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:03.428506 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d76f8-c628-4af2-b1da-86cde020cc14 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:03.430545 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 476/955] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:01:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:01:03.436975 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4de5b4-7bcb-4f22-b11b-3c43b40ade4a None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:03.437930 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4de5b4-7bcb-4f22-b11b-3c43b40ade4a None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:03.438460 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4de5b4-7bcb-4f22-b11b-3c43b40ade4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:03.459337 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-8e4de5b4-7bcb-4f22-b11b-3c43b40ade4a None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:01:03.461245 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-8e4de5b4-7bcb-4f22-b11b-3c43b40ade4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:01:03.548419 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 480/956] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:01:03 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 16:01:03.817314 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76d76f8-c628-4af2-b1da-86cde020cc14 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:03.925331 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed7508e-05eb-4e24-b49d-3f8e4260a230 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:03.925965 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed7508e-05eb-4e24-b49d-3f8e4260a230 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:03.926489 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed7508e-05eb-4e24-b49d-3f8e4260a230 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:03.928282 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 481/957] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:01:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:01:03.933339 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76d76f8-c628-4af2-b1da-86cde020cc14 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:03.934996 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e01f233-f24f-4aeb-a88a-fedbda9bff1d None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:03.935481 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e01f233-f24f-4aeb-a88a-fedbda9bff1d None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:03.935949 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e01f233-f24f-4aeb-a88a-fedbda9bff1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:03.955042 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-9e01f233-f24f-4aeb-a88a-fedbda9bff1d None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:01:03.956564 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-9e01f233-f24f-4aeb-a88a-fedbda9bff1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:01:03.969044 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:03.978393 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90764e7-e007-440b-a489-15ee6818399c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:03.979221 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90764e7-e007-440b-a489-15ee6818399c None admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:03.979746 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90764e7-e007-440b-a489-15ee6818399c None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:03.980017 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90764e7-e007-440b-a489-15ee6818399c None admin] PATH_INFO: `/v3/services` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:03.980730 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90764e7-e007-440b-a489-15ee6818399c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:03.982146 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90764e7-e007-440b-a489-15ee6818399c None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:03.998413 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 477/958] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Sep 19 16:01:03 2022] POST /identity/v3/services => generated 242 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:01:04.066286 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 482/959] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:01:03 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 16:01:04.329356 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90764e7-e007-440b-a489-15ee6818399c None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:04.367168 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:04.442026 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90764e7-e007-440b-a489-15ee6818399c None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:04.480939 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979cb791-2fb3-46ad-851d-0bb64fb26f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:04.481954 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979cb791-2fb3-46ad-851d-0bb64fb26f10 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:04.482442 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979cb791-2fb3-46ad-851d-0bb64fb26f10 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:04.482894 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979cb791-2fb3-46ad-851d-0bb64fb26f10 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:04.483685 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979cb791-2fb3-46ad-851d-0bb64fb26f10 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:04.485713 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979cb791-2fb3-46ad-851d-0bb64fb26f10 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:04.496703 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-979cb791-2fb3-46ad-851d-0bb64fb26f10 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 16:01:04.496703 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:01:04.496703 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 16:01:04.496703 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 16:01:04.496703 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 16:01:04.496703 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 16:01:04.496703 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 16:01:04.496703 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 16:01:04.496703 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 16:01:04.496703 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:01:04.496703 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 16:01:04.496703 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:01:04.496703 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:01:04.496703 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:01:04.496703 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:01:04.496703 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:01:04.496703 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:01:04.497926 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:01:04.497926 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:01:04.497926 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:01:04.497926 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:01:04.497926 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:01:04.497926 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:01:04.497926 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 16:01:04.497926 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 16:01:04.497926 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 16:01:04.497926 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 16:01:04.497926 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:01:04.497926 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:01:04.497926 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:01:04.497926 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:01:04.497926 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:01:04.497926 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:01:04.498827 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:01:04.498827 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:01:04.498827 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:01:04.498827 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:01:04.498827 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:01:04.498827 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:01:04.498827 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:01:04.498827 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:01:04.498827 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:01:04.498827 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:01:04.498827 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 16:01:04.498827 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 16:01:04.498827 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 16:01:04.498827 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:01:04.501493 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 478/960] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Sep 19 16:01:04 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:01:04.507582 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e01f233-f24f-4aeb-a88a-fedbda9bff1d None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:04.626881 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e01f233-f24f-4aeb-a88a-fedbda9bff1d None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:04.628982 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed4da96-9a5f-4b88-b443-242f95335da7 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:04.629504 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed4da96-9a5f-4b88-b443-242f95335da7 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:04.629987 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed4da96-9a5f-4b88-b443-242f95335da7 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:04.631918 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 479/961] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:01:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:01:04.639556 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8fb6f-cd8b-4e85-b080-9cc776885ba8 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:04.640154 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8fb6f-cd8b-4e85-b080-9cc776885ba8 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:04.640632 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8fb6f-cd8b-4e85-b080-9cc776885ba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:04.663305 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-f4f8fb6f-cd8b-4e85-b080-9cc776885ba8 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:01:04.664914 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-f4f8fb6f-cd8b-4e85-b080-9cc776885ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:01:04.665202 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbd5779-aa58-4785-8f1a-ad89cef90685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:04.665905 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbd5779-aa58-4785-8f1a-ad89cef90685 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:04.666183 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbd5779-aa58-4785-8f1a-ad89cef90685 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:04.666529 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbd5779-aa58-4785-8f1a-ad89cef90685 None admin] PATH_INFO: `/v3/services` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:04.667426 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbd5779-aa58-4785-8f1a-ad89cef90685 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:04.668256 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbd5779-aa58-4785-8f1a-ad89cef90685 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:04.669643 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbd5779-aa58-4785-8f1a-ad89cef90685 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:04.675524 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 483/962] 10.222.0.57 () {56 vars in 1122 bytes} [Mon Sep 19 16:01:04 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:04.684085 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbd5779-aa58-4785-8f1a-ad89cef90685 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:04.690071 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:04.793422 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 480/963] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:01:04 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 16:01:04.845661 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbd5779-aa58-4785-8f1a-ad89cef90685 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:04.888899 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cff448-eb36-446c-99c0-43ddfc521d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:04.889448 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cff448-eb36-446c-99c0-43ddfc521d0a None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:04.889726 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cff448-eb36-446c-99c0-43ddfc521d0a None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:04.889994 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cff448-eb36-446c-99c0-43ddfc521d0a None admin] PATH_INFO: `/v3/services` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:04.890717 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cff448-eb36-446c-99c0-43ddfc521d0a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:04.891020 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cff448-eb36-446c-99c0-43ddfc521d0a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:04.892374 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cff448-eb36-446c-99c0-43ddfc521d0a None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:04.898123 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 484/964] 10.222.0.57 () {56 vars in 1122 bytes} [Mon Sep 19 16:01:04 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:04.903894 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f8fb6f-cd8b-4e85-b080-9cc776885ba8 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:04.941017 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:04.954906 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581ac017-1c59-47da-b280-4f2714c56b8b None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:04.955217 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581ac017-1c59-47da-b280-4f2714c56b8b None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:04.955451 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581ac017-1c59-47da-b280-4f2714c56b8b None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:04.957717 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 485/965] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:01:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:01:04.962443 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c182cb-8a01-4691-9e1f-58e890d9997e None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:04.963049 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c182cb-8a01-4691-9e1f-58e890d9997e None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:04.963529 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c182cb-8a01-4691-9e1f-58e890d9997e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:04.986904 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-50c182cb-8a01-4691-9e1f-58e890d9997e None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:01:04.988573 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-50c182cb-8a01-4691-9e1f-58e890d9997e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:01:05.033872 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f8fb6f-cd8b-4e85-b080-9cc776885ba8 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:05.076787 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:05.089008 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ecb0a1-4d75-4d61-862b-632a303af0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:05.089904 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ecb0a1-4d75-4d61-862b-632a303af0cb None admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:05.090403 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ecb0a1-4d75-4d61-862b-632a303af0cb None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:05.090762 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 486/966] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:01:04 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 16:01:05.094800 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ecb0a1-4d75-4d61-862b-632a303af0cb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:05.095767 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ecb0a1-4d75-4d61-862b-632a303af0cb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:05.097511 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ecb0a1-4d75-4d61-862b-632a303af0cb None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:05.110813 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c182cb-8a01-4691-9e1f-58e890d9997e None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:05.122486 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 481/967] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:01:04 2022] POST /identity/v3/endpoints => generated 349 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:01:05.239368 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c182cb-8a01-4691-9e1f-58e890d9997e None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:05.286521 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d36ed3-0be0-4bd0-afa7-92d45294a147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:05.287159 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d36ed3-0be0-4bd0-afa7-92d45294a147 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:05.287432 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d36ed3-0be0-4bd0-afa7-92d45294a147 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:05.287698 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d36ed3-0be0-4bd0-afa7-92d45294a147 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:05.300627 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d36ed3-0be0-4bd0-afa7-92d45294a147 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 16:01:05.305554 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-b9d36ed3-0be0-4bd0-afa7-92d45294a147 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 16:01:05.305554 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:01:05.305554 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:01:05.305554 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:01:05.305554 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:01:05.305554 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:01:05.305554 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:01:05.305554 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:01:05.305554 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:01:05.305554 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:01:05.305554 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:01:05.305554 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:01:05.305554 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 16:01:05.305554 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 16:01:05.305554 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 16:01:05.305554 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 16:01:05.306556 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 16:01:05.306556 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 16:01:05.306556 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 16:01:05.306556 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 16:01:05.306556 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:01:05.306556 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:01:05.306556 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:01:05.306556 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:01:05.306556 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:01:05.306556 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:01:05.306556 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:01:05.306556 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:01:05.306556 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:01:05.306556 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:01:05.306556 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:01:05.306556 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:01:05.307387 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:01:05.307387 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:01:05.307387 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:01:05.307387 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:01:05.307387 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 16:01:05.307387 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 16:01:05.307387 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 16:01:05.307387 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 16:01:05.307387 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 16:01:05.307387 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 16:01:05.307387 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 16:01:05.307387 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:01:05.307387 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 487/968] 10.222.0.57 () {56 vars in 1120 bytes} [Mon Sep 19 16:01:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 16:01:05.314329 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ecb0a1-4d75-4d61-862b-632a303af0cb None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:05.386024 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d36ed3-0be0-4bd0-afa7-92d45294a147 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:05.406781 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:05.430651 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ecb0a1-4d75-4d61-862b-632a303af0cb None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:05.447573 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:05.478632 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48152e6-9966-4ce0-91fc-90f3e3fdfdbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:05.479324 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48152e6-9966-4ce0-91fc-90f3e3fdfdbe None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:05.479518 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48152e6-9966-4ce0-91fc-90f3e3fdfdbe None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:05.479766 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48152e6-9966-4ce0-91fc-90f3e3fdfdbe None admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:05.480451 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48152e6-9966-4ce0-91fc-90f3e3fdfdbe None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:05.480748 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48152e6-9966-4ce0-91fc-90f3e3fdfdbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:05.482091 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48152e6-9966-4ce0-91fc-90f3e3fdfdbe None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:05.490877 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 482/969] 10.222.0.57 () {56 vars in 1115 bytes} [Mon Sep 19 16:01:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:01:05.496222 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48152e6-9966-4ce0-91fc-90f3e3fdfdbe None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:05.511990 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d36ed3-0be0-4bd0-afa7-92d45294a147 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:05.550168 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16fbe66-5744-4b67-bbc5-565b28460e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:05.551094 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16fbe66-5744-4b67-bbc5-565b28460e0d None admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:05.551578 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16fbe66-5744-4b67-bbc5-565b28460e0d None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:05.551993 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16fbe66-5744-4b67-bbc5-565b28460e0d None admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:05.552862 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16fbe66-5744-4b67-bbc5-565b28460e0d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:05.554511 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16fbe66-5744-4b67-bbc5-565b28460e0d None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:05.572920 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.sql.core [None req-a16fbe66-5744-4b67-bbc5-565b28460e0d None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 19 16:01:05.572920 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 19 16:01:05.572920 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [parameters: {'user_id': '2ed6763280af4a52a8526c2c5fb2a8b1', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 19 16:01:05.572920 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81617) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 19 16:01:05.581617 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-a16fbe66-5744-4b67-bbc5-565b28460e0d None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 19 16:01:05.581617 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:01:05.581617 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 19 16:01:05.581617 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 19 16:01:05.581617 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 19 16:01:05.581617 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 19 16:01:05.581617 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 19 16:01:05.581617 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application result = self._query(query) Sep 19 16:01:05.581617 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 19 16:01:05.581617 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application conn.query(q) Sep 19 16:01:05.581617 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 19 16:01:05.581617 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 19 16:01:05.581617 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 19 16:01:05.581617 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application result.read() Sep 19 16:01:05.581617 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 19 16:01:05.582561 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 19 16:01:05.582561 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 19 16:01:05.582561 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 19 16:01:05.582561 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 19 16:01:05.582561 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 19 16:01:05.582561 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 19 16:01:05.582561 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 19 16:01:05.582561 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 19 16:01:05.582561 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:01:05.582561 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 19 16:01:05.582561 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:01:05.582561 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:01:05.582561 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 19 16:01:05.582561 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 19 16:01:05.582561 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 19 16:01:05.582561 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 19 16:01:05.582561 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 19 16:01:05.582561 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application next(self.gen) Sep 19 16:01:05.583511 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 19 16:01:05.583511 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application yield resource Sep 19 16:01:05.583511 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 19 16:01:05.583511 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application next(self.gen) Sep 19 16:01:05.583511 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 19 16:01:05.583511 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application self.session.rollback() Sep 19 16:01:05.583511 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 19 16:01:05.583511 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application self.force_reraise() Sep 19 16:01:05.583511 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 19 16:01:05.583511 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise self.value Sep 19 16:01:05.583511 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 19 16:01:05.583511 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 19 16:01:05.583511 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 19 16:01:05.583511 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application session.commit() Sep 19 16:01:05.583511 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 19 16:01:05.583511 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 19 16:01:05.583511 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 19 16:01:05.584387 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application self._prepare_impl() Sep 19 16:01:05.584387 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 19 16:01:05.584387 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application self.session.flush() Sep 19 16:01:05.584387 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 19 16:01:05.584387 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application self._flush(objects) Sep 19 16:01:05.584387 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 19 16:01:05.584387 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 19 16:01:05.584387 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 19 16:01:05.584387 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application compat.raise_( Sep 19 16:01:05.584387 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 19 16:01:05.584387 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception Sep 19 16:01:05.584387 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 19 16:01:05.584387 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application flush_context.execute() Sep 19 16:01:05.584387 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 19 16:01:05.584387 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rec.execute(self) Sep 19 16:01:05.584387 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 19 16:01:05.584387 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 19 16:01:05.585263 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 19 16:01:05.585263 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 19 16:01:05.585263 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 19 16:01:05.585263 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 19 16:01:05.585263 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 19 16:01:05.585263 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 19 16:01:05.585263 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 19 16:01:05.585263 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 19 16:01:05.585263 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 19 16:01:05.585263 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 19 16:01:05.585263 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 19 16:01:05.585263 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 19 16:01:05.585263 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 19 16:01:05.585263 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 19 16:01:05.585263 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 19 16:01:05.585263 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception Sep 19 16:01:05.586181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 19 16:01:05.586181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 19 16:01:05.586181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 19 16:01:05.586181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 19 16:01:05.586181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 19 16:01:05.586181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application result = self._query(query) Sep 19 16:01:05.586181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 19 16:01:05.586181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application conn.query(q) Sep 19 16:01:05.586181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 19 16:01:05.586181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 19 16:01:05.586181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 19 16:01:05.586181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application result.read() Sep 19 16:01:05.586181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 19 16:01:05.586181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 19 16:01:05.586181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 19 16:01:05.586181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 19 16:01:05.587056 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 19 16:01:05.587056 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 19 16:01:05.587056 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 19 16:01:05.587056 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 19 16:01:05.587056 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 19 16:01:05.587056 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 19 16:01:05.587056 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [parameters: {'user_id': '2ed6763280af4a52a8526c2c5fb2a8b1', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 19 16:01:05.587056 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 19 16:01:05.587056 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:01:05.587056 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 16:01:05.587056 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:01:05.587056 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:01:05.587056 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:01:05.587056 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:01:05.587056 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:01:05.588012 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:01:05.588012 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:01:05.588012 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:01:05.588012 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:01:05.588012 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:01:05.588012 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:01:05.588012 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:01:05.588012 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 19 16:01:05.588012 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 19 16:01:05.588012 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:01:05.588012 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:01:05.588012 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 16:01:05.588012 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:01:05.588012 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 16:01:05.588012 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:01:05.588012 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 19 16:01:05.588012 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 19 16:01:05.588893 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 19 16:01:05.588893 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 19 16:01:05.588893 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 19 16:01:05.588893 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 19 16:01:05.588893 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 19 16:01:05.588893 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:01:05.591694 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 488/970] 10.222.0.57 () {60 vars in 1131 bytes} [Mon Sep 19 16:01:05 2022] POST /identity/v3/users => generated 160 bytes in 206 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 19 16:01:05.599131 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16fbe66-5744-4b67-bbc5-565b28460e0d None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:05.617918 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48152e6-9966-4ce0-91fc-90f3e3fdfdbe None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:05.656771 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78044c54-8a18-41ec-884c-a67b6914ce2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:05.657578 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78044c54-8a18-41ec-884c-a67b6914ce2d None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:05.657863 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78044c54-8a18-41ec-884c-a67b6914ce2d None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:05.658154 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78044c54-8a18-41ec-884c-a67b6914ce2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:05.671311 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78044c54-8a18-41ec-884c-a67b6914ce2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:05.672822 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78044c54-8a18-41ec-884c-a67b6914ce2d None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:05.681187 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:05.690675 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-78044c54-8a18-41ec-884c-a67b6914ce2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:01:05.690675 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:01:05.690675 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:01:05.690675 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:01:05.690675 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:01:05.690675 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:01:05.690675 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:01:05.690675 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:01:05.690675 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:01:05.690675 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:01:05.690675 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:01:05.690675 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:01:05.690675 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:01:05.690675 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:01:05.690675 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:01:05.690675 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:01:05.691660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:01:05.691660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:01:05.691660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:01:05.691660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:01:05.691660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:01:05.691660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:01:05.691660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:01:05.691660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:01:05.691660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:01:05.691660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:01:05.691660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:01:05.691660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:01:05.691660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:01:05.691660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:01:05.691660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:01:05.691660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:01:05.691660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:01:05.692537 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:01:05.692537 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:01:05.692537 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:01:05.692537 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 483/971] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:01:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 16:01:05.698426 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78044c54-8a18-41ec-884c-a67b6914ce2d None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:05.787859 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16fbe66-5744-4b67-bbc5-565b28460e0d None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:05.804802 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:05.826085 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78044c54-8a18-41ec-884c-a67b6914ce2d None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:05.828904 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1423832-8293-4f74-a50f-5b418faacad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:05.829846 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1423832-8293-4f74-a50f-5b418faacad5 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:05.830331 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1423832-8293-4f74-a50f-5b418faacad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:05.830834 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1423832-8293-4f74-a50f-5b418faacad5 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:05.838913 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1423832-8293-4f74-a50f-5b418faacad5 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:05.840943 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1423832-8293-4f74-a50f-5b418faacad5 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:05.853780 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-e1423832-8293-4f74-a50f-5b418faacad5 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 19 16:01:05.853780 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:01:05.853780 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:01:05.853780 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:01:05.853780 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:01:05.853780 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:01:05.853780 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:01:05.853780 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:01:05.853780 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:01:05.853780 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:01:05.853780 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:01:05.853780 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:01:05.853780 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 16:01:05.853780 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 16:01:05.853780 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 16:01:05.853780 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 16:01:05.854815 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:01:05.854815 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:01:05.854815 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 16:01:05.854815 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:01:05.854815 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 16:01:05.854815 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:01:05.854815 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:01:05.854815 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:01:05.854815 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:01:05.854815 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:01:05.854815 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:01:05.854815 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:01:05.854815 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:01:05.854815 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:01:05.854815 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:01:05.854815 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:01:05.854815 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:01:05.854815 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:01:05.855801 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:01:05.855801 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:01:05.855801 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 16:01:05.855801 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 16:01:05.855801 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 16:01:05.855801 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 16:01:05.855801 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 16:01:05.855801 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 16:01:05.855801 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 19 16:01:05.855801 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:01:05.858453 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 489/972] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 16:01:05 2022] GET /identity/v3/users/cinder => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 16:01:05.863924 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1423832-8293-4f74-a50f-5b418faacad5 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:05.865090 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2156e6d-6841-4ad5-bcbc-379332bab0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:05.865707 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2156e6d-6841-4ad5-bcbc-379332bab0ab None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:05.865989 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2156e6d-6841-4ad5-bcbc-379332bab0ab None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:05.866264 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2156e6d-6841-4ad5-bcbc-379332bab0ab None admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:05.866960 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2156e6d-6841-4ad5-bcbc-379332bab0ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:05.867264 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2156e6d-6841-4ad5-bcbc-379332bab0ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:05.868752 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2156e6d-6841-4ad5-bcbc-379332bab0ab None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:05.888536 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 484/973] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:01:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:01:05.895772 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2156e6d-6841-4ad5-bcbc-379332bab0ab None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:05.977798 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:06.003152 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1423832-8293-4f74-a50f-5b418faacad5 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:06.032455 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2156e6d-6841-4ad5-bcbc-379332bab0ab None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:06.050764 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a5eea8-0a08-4bc2-8de0-c6661f9a7f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:06.051868 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a5eea8-0a08-4bc2-8de0-c6661f9a7f34 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:06.052305 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a5eea8-0a08-4bc2-8de0-c6661f9a7f34 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:06.052968 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a5eea8-0a08-4bc2-8de0-c6661f9a7f34 None admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:06.054217 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a5eea8-0a08-4bc2-8de0-c6661f9a7f34 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:06.054693 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a5eea8-0a08-4bc2-8de0-c6661f9a7f34 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:06.056268 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a5eea8-0a08-4bc2-8de0-c6661f9a7f34 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:06.076463 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fa2ddd-96fe-4c9c-82db-9afa79409c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:06.077359 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fa2ddd-96fe-4c9c-82db-9afa79409c8d None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:06.077784 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fa2ddd-96fe-4c9c-82db-9afa79409c8d None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:06.078214 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fa2ddd-96fe-4c9c-82db-9afa79409c8d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:06.087157 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fa2ddd-96fe-4c9c-82db-9afa79409c8d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:06.088853 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fa2ddd-96fe-4c9c-82db-9afa79409c8d None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:06.099520 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 490/974] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Sep 19 16:01:05 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:01:06.103934 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-c8fa2ddd-96fe-4c9c-82db-9afa79409c8d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 16:01:06.103934 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:01:06.103934 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:01:06.103934 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:01:06.103934 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:01:06.103934 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:01:06.103934 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:01:06.103934 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:01:06.103934 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:01:06.103934 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:01:06.103934 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:01:06.103934 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:01:06.103934 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 16:01:06.103934 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 16:01:06.103934 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 16:01:06.103934 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 16:01:06.104948 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:01:06.104948 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:01:06.104948 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 16:01:06.104948 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:01:06.104948 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 16:01:06.104948 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:01:06.104948 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:01:06.104948 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:01:06.104948 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:01:06.104948 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:01:06.104948 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:01:06.104948 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:01:06.104948 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:01:06.104948 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:01:06.104948 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:01:06.104948 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:01:06.104948 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:01:06.104948 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:01:06.105988 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:01:06.105988 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:01:06.105988 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 16:01:06.105988 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 16:01:06.105988 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 16:01:06.105988 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 16:01:06.105988 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 16:01:06.105988 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 16:01:06.105988 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 16:01:06.105988 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:01:06.113128 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 485/975] 10.222.0.57 () {56 vars in 1117 bytes} [Mon Sep 19 16:01:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:01:06.114995 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a5eea8-0a08-4bc2-8de0-c6661f9a7f34 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:06.157501 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:06.247995 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a5eea8-0a08-4bc2-8de0-c6661f9a7f34 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:06.287597 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d2f694-3397-425d-ab41-6e05c6901668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:06.288445 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d2f694-3397-425d-ab41-6e05c6901668 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:06.288874 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d2f694-3397-425d-ab41-6e05c6901668 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:06.289309 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d2f694-3397-425d-ab41-6e05c6901668 None admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:06.290186 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d2f694-3397-425d-ab41-6e05c6901668 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:06.290679 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d2f694-3397-425d-ab41-6e05c6901668 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:06.293083 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d2f694-3397-425d-ab41-6e05c6901668 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:06.316739 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 491/976] 10.222.0.57 () {56 vars in 1149 bytes} [Mon Sep 19 16:01:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:06.320839 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fa2ddd-96fe-4c9c-82db-9afa79409c8d None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:06.357146 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9e8c1e-03fd-476f-9a19-237d1e4af22c None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:06.357465 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9e8c1e-03fd-476f-9a19-237d1e4af22c None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:06.357706 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9e8c1e-03fd-476f-9a19-237d1e4af22c None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:06.359291 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 492/977] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:01:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:01:06.366816 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8eb90a-9fce-48e4-a342-3f7db905bd5f None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:06.367098 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8eb90a-9fce-48e4-a342-3f7db905bd5f None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:06.367376 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8eb90a-9fce-48e4-a342-3f7db905bd5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:06.393730 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-2e8eb90a-9fce-48e4-a342-3f7db905bd5f None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:01:06.395939 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-2e8eb90a-9fce-48e4-a342-3f7db905bd5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:01:06.441563 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fa2ddd-96fe-4c9c-82db-9afa79409c8d None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:06.461627 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:06.475545 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 493/978] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:01:06 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 16:01:06.483578 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8003f2cf-bd3d-4297-899e-8bdd7b2ea272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:06.484271 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8003f2cf-bd3d-4297-899e-8bdd7b2ea272 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:06.484655 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8003f2cf-bd3d-4297-899e-8bdd7b2ea272 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:06.485000 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8003f2cf-bd3d-4297-899e-8bdd7b2ea272 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:06.556091 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8003f2cf-bd3d-4297-899e-8bdd7b2ea272 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:06.557453 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8003f2cf-bd3d-4297-899e-8bdd7b2ea272 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:06.661749 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 486/979] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 19 16:01:06 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 16:01:06.663650 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8eb90a-9fce-48e4-a342-3f7db905bd5f None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:06.687077 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:06.752335 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8003f2cf-bd3d-4297-899e-8bdd7b2ea272 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:06.773844 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:06.775808 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8eb90a-9fce-48e4-a342-3f7db905bd5f None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:06.817134 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6177a71b-1a16-44fe-a83a-119056fd17f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:06.817685 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6177a71b-1a16-44fe-a83a-119056fd17f0 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:06.817963 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6177a71b-1a16-44fe-a83a-119056fd17f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:06.818237 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6177a71b-1a16-44fe-a83a-119056fd17f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:06.830186 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6177a71b-1a16-44fe-a83a-119056fd17f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:06.831672 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6177a71b-1a16-44fe-a83a-119056fd17f0 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:06.847266 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-6177a71b-1a16-44fe-a83a-119056fd17f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:01:06.847266 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:01:06.847266 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:01:06.847266 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:01:06.847266 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:01:06.847266 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:01:06.847266 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:01:06.847266 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:01:06.847266 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:01:06.847266 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:01:06.847266 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:01:06.847266 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:01:06.847266 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:01:06.847266 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:01:06.847266 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:01:06.847266 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:01:06.850572 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:01:06.850572 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:01:06.850572 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:01:06.850572 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:01:06.850572 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:01:06.850572 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:01:06.850572 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:01:06.850572 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:01:06.850572 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:01:06.850572 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:01:06.850572 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:01:06.850572 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:01:06.850572 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:01:06.850572 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:01:06.850572 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:01:06.850572 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:01:06.850572 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:01:06.851523 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:01:06.851523 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:01:06.851523 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:01:06.851523 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 494/980] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:01:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:01:06.855524 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6177a71b-1a16-44fe-a83a-119056fd17f0 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:06.872432 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8003f2cf-bd3d-4297-899e-8bdd7b2ea272 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:06.911154 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc180f7-339d-46c9-b829-cea57cf5b8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:06.912039 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc180f7-339d-46c9-b829-cea57cf5b8ee None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:06.912496 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc180f7-339d-46c9-b829-cea57cf5b8ee None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:06.912927 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc180f7-339d-46c9-b829-cea57cf5b8ee None admin] PATH_INFO: `/v3/services/compute` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:06.913676 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc180f7-339d-46c9-b829-cea57cf5b8ee None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:06.915330 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc180f7-339d-46c9-b829-cea57cf5b8ee None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:06.925655 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-6cc180f7-339d-46c9-b829-cea57cf5b8ee None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 19 16:01:06.925655 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:01:06.925655 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 16:01:06.925655 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 16:01:06.925655 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 16:01:06.925655 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 16:01:06.925655 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 16:01:06.925655 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 16:01:06.925655 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 19 16:01:06.925655 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:01:06.925655 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 16:01:06.925655 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:01:06.925655 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:01:06.925655 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:01:06.925655 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:01:06.925655 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:01:06.925655 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:01:06.926801 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:01:06.926801 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:01:06.926801 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:01:06.926801 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:01:06.926801 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:01:06.926801 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:01:06.926801 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 16:01:06.926801 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 16:01:06.926801 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 16:01:06.926801 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 16:01:06.926801 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:01:06.926801 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:01:06.926801 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:01:06.926801 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:01:06.926801 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:01:06.926801 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:01:06.927629 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:01:06.927629 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:01:06.927629 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:01:06.927629 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:01:06.927629 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:01:06.927629 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:01:06.927629 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:01:06.927629 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:01:06.927629 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:01:06.927629 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:01:06.927629 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 16:01:06.927629 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 16:01:06.927629 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 19 16:01:06.927629 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:01:06.927629 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 487/981] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 19 16:01:06 2022] GET /identity/v3/services/compute => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 16:01:06.935924 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc180f7-339d-46c9-b829-cea57cf5b8ee None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:06.938590 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:06.971489 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6177a71b-1a16-44fe-a83a-119056fd17f0 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:07.008941 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24ad730-7e96-4b59-9f16-a9e7965052f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:07.009570 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ad730-7e96-4b59-9f16-a9e7965052f4 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:07.009840 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ad730-7e96-4b59-9f16-a9e7965052f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:07.010108 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ad730-7e96-4b59-9f16-a9e7965052f4 None admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:07.010802 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24ad730-7e96-4b59-9f16-a9e7965052f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:07.011123 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24ad730-7e96-4b59-9f16-a9e7965052f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:07.012558 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24ad730-7e96-4b59-9f16-a9e7965052f4 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:07.026748 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 495/982] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:01:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:01:07.032555 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24ad730-7e96-4b59-9f16-a9e7965052f4 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:07.036364 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:07.064208 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc180f7-339d-46c9-b829-cea57cf5b8ee None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:07.123741 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd510c1-aac7-4a7f-9954-9058bfcbb4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:07.124666 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd510c1-aac7-4a7f-9954-9058bfcbb4a4 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:07.125132 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd510c1-aac7-4a7f-9954-9058bfcbb4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:07.125605 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd510c1-aac7-4a7f-9954-9058bfcbb4a4 None admin] PATH_INFO: `/v3/services` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:07.126738 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd510c1-aac7-4a7f-9954-9058bfcbb4a4 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:07.127266 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd510c1-aac7-4a7f-9954-9058bfcbb4a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:07.129719 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd510c1-aac7-4a7f-9954-9058bfcbb4a4 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:07.142533 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 488/983] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Sep 19 16:01:06 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:01:07.143543 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd510c1-aac7-4a7f-9954-9058bfcbb4a4 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:07.147969 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24ad730-7e96-4b59-9f16-a9e7965052f4 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:07.184753 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:07.191991 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd20adc6-7554-4224-a159-f27a9b1a5b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:07.192632 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd20adc6-7554-4224-a159-f27a9b1a5b85 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:07.192906 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd20adc6-7554-4224-a159-f27a9b1a5b85 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:07.193176 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd20adc6-7554-4224-a159-f27a9b1a5b85 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:07.201041 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd20adc6-7554-4224-a159-f27a9b1a5b85 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:07.202648 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd20adc6-7554-4224-a159-f27a9b1a5b85 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:07.215212 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-bd20adc6-7554-4224-a159-f27a9b1a5b85 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:01:07.215212 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:01:07.215212 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:01:07.215212 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:01:07.215212 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:01:07.215212 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:01:07.215212 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:01:07.215212 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:01:07.215212 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:01:07.215212 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:01:07.215212 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:01:07.215212 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:01:07.215212 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 16:01:07.215212 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 16:01:07.215212 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 16:01:07.216152 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 16:01:07.216152 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:01:07.216152 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:01:07.216152 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:01:07.216152 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:01:07.216152 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:01:07.216152 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:01:07.216152 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:01:07.216152 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:01:07.216152 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:01:07.216152 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:01:07.216152 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:01:07.216152 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:01:07.216152 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:01:07.216152 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:01:07.216152 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:01:07.216152 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:01:07.217178 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 16:01:07.217178 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 16:01:07.217178 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 16:01:07.217178 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 16:01:07.217178 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 16:01:07.217178 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 16:01:07.217178 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:01:07.217178 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:01:07.217178 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 496/984] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 19 16:01:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 16:01:07.221915 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd20adc6-7554-4224-a159-f27a9b1a5b85 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:07.270662 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd510c1-aac7-4a7f-9954-9058bfcbb4a4 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:07.320421 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c72623-8593-473e-8b08-3846788b4a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:07.321126 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c72623-8593-473e-8b08-3846788b4a1b None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:07.321497 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c72623-8593-473e-8b08-3846788b4a1b None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:07.321841 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c72623-8593-473e-8b08-3846788b4a1b None admin] PATH_INFO: `/v3/services` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:07.322650 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c72623-8593-473e-8b08-3846788b4a1b None admin] RBAC: Adding query filter params (type=compute) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:07.323067 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c72623-8593-473e-8b08-3846788b4a1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:07.324515 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c72623-8593-473e-8b08-3846788b4a1b None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:07.334778 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 489/985] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Sep 19 16:01:07 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:01:07.335959 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c72623-8593-473e-8b08-3846788b4a1b None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:07.350226 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:07.353836 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd20adc6-7554-4224-a159-f27a9b1a5b85 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:07.406223 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ec8ee8-bbff-40d3-8476-525e7aef3555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:07.406764 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec8ee8-bbff-40d3-8476-525e7aef3555 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:07.407064 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec8ee8-bbff-40d3-8476-525e7aef3555 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:07.407321 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec8ee8-bbff-40d3-8476-525e7aef3555 None admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:07.408027 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ec8ee8-bbff-40d3-8476-525e7aef3555 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:07.409983 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ec8ee8-bbff-40d3-8476-525e7aef3555 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:07.409983 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ec8ee8-bbff-40d3-8476-525e7aef3555 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:07.423715 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 497/986] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 19 16:01:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:01:07.425047 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69e918-4619-4b13-8f04-5a904ece91e5 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:07.425331 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69e918-4619-4b13-8f04-5a904ece91e5 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:07.425630 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69e918-4619-4b13-8f04-5a904ece91e5 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:07.427177 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 498/987] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:01:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:01:07.430216 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f69e918-4619-4b13-8f04-5a904ece91e5 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:07.456445 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c72623-8593-473e-8b08-3846788b4a1b None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:07.493195 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:07.500749 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fee2bc-e7ca-4ac5-b375-2026e39330c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:07.501404 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fee2bc-e7ca-4ac5-b375-2026e39330c2 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:07.501613 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fee2bc-e7ca-4ac5-b375-2026e39330c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:07.501885 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fee2bc-e7ca-4ac5-b375-2026e39330c2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:07.502605 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fee2bc-e7ca-4ac5-b375-2026e39330c2 None admin] RBAC: Adding query filter params (interface=public, service_id=8bef90a659544bec815b4ce1af6ef961, region_id=RegionOne) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:07.502915 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fee2bc-e7ca-4ac5-b375-2026e39330c2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:07.504264 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fee2bc-e7ca-4ac5-b375-2026e39330c2 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:07.509588 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 490/988] 10.222.0.57 () {56 vars in 1251 bytes} [Mon Sep 19 16:01:07 2022] GET /identity/v3/endpoints?service_id=8bef90a659544bec815b4ce1af6ef961&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:07.510881 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd192d1-eec4-4041-a163-043ddfaa36bf None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:07.511163 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd192d1-eec4-4041-a163-043ddfaa36bf None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:07.511466 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd192d1-eec4-4041-a163-043ddfaa36bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:07.531641 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-abd192d1-eec4-4041-a163-043ddfaa36bf None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:01:07.532993 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-abd192d1-eec4-4041-a163-043ddfaa36bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:01:07.551471 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f69e918-4619-4b13-8f04-5a904ece91e5 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:07.587249 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9328c509-3e23-47d8-bb45-ce2d56c35f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:07.587860 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9328c509-3e23-47d8-bb45-ce2d56c35f6f None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:07.588141 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9328c509-3e23-47d8-bb45-ce2d56c35f6f None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:07.588414 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9328c509-3e23-47d8-bb45-ce2d56c35f6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:07.589136 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9328c509-3e23-47d8-bb45-ce2d56c35f6f None admin] RBAC: Adding query filter params (role.id=c30e151440d44fd18e607056b4d41ec7, scope.project.id=be03e5300b3343169f35c643c93d1e06, user.id=7222bd3b313b47e5a4fa0ebc9ff0cdd5) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:07.590074 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9328c509-3e23-47d8-bb45-ce2d56c35f6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:07.591501 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9328c509-3e23-47d8-bb45-ce2d56c35f6f None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:07.597953 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 499/989] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 19 16:01:07 2022] GET /identity/v3/role_assignments?user.id=7222bd3b313b47e5a4fa0ebc9ff0cdd5&scope.project.id=be03e5300b3343169f35c643c93d1e06&role.id=c30e151440d44fd18e607056b4d41ec7 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:07.598864 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9328c509-3e23-47d8-bb45-ce2d56c35f6f None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:07.620081 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 491/990] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:01:07 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 16:01:07.736592 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9328c509-3e23-47d8-bb45-ce2d56c35f6f None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:07.756160 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:07.784809 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705e9351-284d-4c63-bd1e-806533ec1fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:07.785450 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e9351-284d-4c63-bd1e-806533ec1fc8 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:07.785762 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e9351-284d-4c63-bd1e-806533ec1fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:07.786081 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e9351-284d-4c63-bd1e-806533ec1fc8 None admin] PATH_INFO: `/v3/services` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:07.786984 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705e9351-284d-4c63-bd1e-806533ec1fc8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:07.788428 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705e9351-284d-4c63-bd1e-806533ec1fc8 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:07.796150 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 500/991] 10.222.0.57 () {56 vars in 1087 bytes} [Mon Sep 19 16:01:07 2022] GET /identity/v3/services => generated 1727 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:01:07.894950 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd192d1-eec4-4041-a163-043ddfaa36bf None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:07.898733 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:08.048907 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd192d1-eec4-4041-a163-043ddfaa36bf None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:08.090994 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b928053-0709-4f1b-b476-7e12d67c0090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:08.091806 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b928053-0709-4f1b-b476-7e12d67c0090 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:08.092175 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b928053-0709-4f1b-b476-7e12d67c0090 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:08.092444 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b928053-0709-4f1b-b476-7e12d67c0090 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:08.093030 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b928053-0709-4f1b-b476-7e12d67c0090 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:08.094421 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b928053-0709-4f1b-b476-7e12d67c0090 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:08.104649 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-0b928053-0709-4f1b-b476-7e12d67c0090 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 16:01:08.104649 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:01:08.104649 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 16:01:08.104649 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 16:01:08.104649 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 16:01:08.104649 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 16:01:08.104649 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 16:01:08.104649 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 16:01:08.104649 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 16:01:08.104649 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:01:08.104649 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 16:01:08.104649 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:01:08.104649 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:01:08.104649 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:01:08.104649 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:01:08.104649 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:01:08.104649 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:01:08.105957 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:01:08.105957 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:01:08.105957 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:01:08.105957 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:01:08.105957 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:01:08.105957 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:01:08.105957 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 16:01:08.105957 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 16:01:08.105957 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 16:01:08.105957 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 16:01:08.105957 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:01:08.105957 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:01:08.105957 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:01:08.105957 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:01:08.105957 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:01:08.105957 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:01:08.107535 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:01:08.107535 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:01:08.107535 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:01:08.107535 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:01:08.107535 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:01:08.107535 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:01:08.107535 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:01:08.107535 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:01:08.107535 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:01:08.107535 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:01:08.107535 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 16:01:08.107535 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 16:01:08.107535 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 16:01:08.107535 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:01:08.107535 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 492/992] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Sep 19 16:01:07 2022] GET /identity/v3/services/object-store => generated 93 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:01:08.112278 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705e9351-284d-4c63-bd1e-806533ec1fc8 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:08.238561 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705e9351-284d-4c63-bd1e-806533ec1fc8 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:08.278388 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd4d431-d1eb-4239-9233-f0b0364ff45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:08.279010 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd4d431-d1eb-4239-9233-f0b0364ff45c None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:08.279291 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd4d431-d1eb-4239-9233-f0b0364ff45c None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:08.279562 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd4d431-d1eb-4239-9233-f0b0364ff45c None admin] PATH_INFO: `/v3/services` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:08.280195 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd4d431-d1eb-4239-9233-f0b0364ff45c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:08.280523 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd4d431-d1eb-4239-9233-f0b0364ff45c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:08.282009 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd4d431-d1eb-4239-9233-f0b0364ff45c None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:08.287597 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 501/993] 10.222.0.57 () {56 vars in 1122 bytes} [Mon Sep 19 16:01:08 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:01:08.293870 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b928053-0709-4f1b-b476-7e12d67c0090 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:08.385460 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:08.437275 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b928053-0709-4f1b-b476-7e12d67c0090 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:08.486012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cb92c9-8a45-46b6-abaa-3ca29975b0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:08.486882 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cb92c9-8a45-46b6-abaa-3ca29975b0f9 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:08.487156 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cb92c9-8a45-46b6-abaa-3ca29975b0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:08.487425 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cb92c9-8a45-46b6-abaa-3ca29975b0f9 None admin] PATH_INFO: `/v3/services` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:08.488316 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cb92c9-8a45-46b6-abaa-3ca29975b0f9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:08.488668 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cb92c9-8a45-46b6-abaa-3ca29975b0f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:08.490927 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cb92c9-8a45-46b6-abaa-3ca29975b0f9 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:08.502419 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 493/994] 10.222.0.57 () {56 vars in 1122 bytes} [Mon Sep 19 16:01:08 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:08.506225 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd4d431-d1eb-4239-9233-f0b0364ff45c None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:08.578715 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:08.647361 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd4d431-d1eb-4239-9233-f0b0364ff45c None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:08.695414 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e621e6-4257-4390-a912-c716a02e4718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:08.696363 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e621e6-4257-4390-a912-c716a02e4718 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:08.696726 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e621e6-4257-4390-a912-c716a02e4718 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:08.697083 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e621e6-4257-4390-a912-c716a02e4718 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:08.698249 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e621e6-4257-4390-a912-c716a02e4718 None admin] RBAC: Adding query filter params (interface=admin, service_id=ff34ff3102a44ff9bb52ee1d14a10487, region_id=RegionOne) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:08.698786 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e621e6-4257-4390-a912-c716a02e4718 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:08.700868 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e621e6-4257-4390-a912-c716a02e4718 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:08.714329 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 502/995] 10.222.0.57 () {56 vars in 1249 bytes} [Mon Sep 19 16:01:08 2022] GET /identity/v3/endpoints?service_id=ff34ff3102a44ff9bb52ee1d14a10487&interface=admin®ion_id=RegionOne => generated 193 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:08.715963 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cb92c9-8a45-46b6-abaa-3ca29975b0f9 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:08.850872 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cb92c9-8a45-46b6-abaa-3ca29975b0f9 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:08.885145 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:08.892933 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e976d0f-10d1-4f05-9c02-c7067124c977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:08.893572 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e976d0f-10d1-4f05-9c02-c7067124c977 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:08.894056 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e976d0f-10d1-4f05-9c02-c7067124c977 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:08.894205 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e976d0f-10d1-4f05-9c02-c7067124c977 None admin] PATH_INFO: `/v3/services` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:08.894765 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e976d0f-10d1-4f05-9c02-c7067124c977 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:08.896170 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e976d0f-10d1-4f05-9c02-c7067124c977 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:08.903948 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 494/996] 10.222.0.57 () {56 vars in 1087 bytes} [Mon Sep 19 16:01:08 2022] GET /identity/v3/services => generated 1727 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:01:09.905685 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6385cd1-974b-4828-beaa-d8d2646f05cc None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:09.906233 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6385cd1-974b-4828-beaa-d8d2646f05cc None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:09.906233 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6385cd1-974b-4828-beaa-d8d2646f05cc None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:09.907744 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 503/997] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:01:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:01:09.912954 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc5ff65-6068-4160-9e76-5f03fa2ecf08 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:09.913407 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc5ff65-6068-4160-9e76-5f03fa2ecf08 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:09.913609 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc5ff65-6068-4160-9e76-5f03fa2ecf08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:09.932736 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-3cc5ff65-6068-4160-9e76-5f03fa2ecf08 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:01:09.934505 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-3cc5ff65-6068-4160-9e76-5f03fa2ecf08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:01:10.018606 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 495/998] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:01:09 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 16:01:10.127793 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17ea52-920a-4fa1-a5a6-2743b53c7ee7 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:10.127988 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17ea52-920a-4fa1-a5a6-2743b53c7ee7 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:10.128258 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17ea52-920a-4fa1-a5a6-2743b53c7ee7 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:10.129803 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 504/999] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:01:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:01:10.135462 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d37937-ceae-4d97-a76c-d5bf793f23f8 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:10.135750 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d37937-ceae-4d97-a76c-d5bf793f23f8 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:10.136025 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d37937-ceae-4d97-a76c-d5bf793f23f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:10.155338 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-56d37937-ceae-4d97-a76c-d5bf793f23f8 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:01:10.156521 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-56d37937-ceae-4d97-a76c-d5bf793f23f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:01:10.246720 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 496/1000] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:01:10 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 16:01:10.305380 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c17ea52-920a-4fa1-a5a6-2743b53c7ee7 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:10.417412 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c17ea52-920a-4fa1-a5a6-2743b53c7ee7 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:10.428790 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:10.461102 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cf369d-312d-4688-aee4-783363723a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:10.461749 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf369d-312d-4688-aee4-783363723a40 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:10.462024 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf369d-312d-4688-aee4-783363723a40 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:10.462292 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf369d-312d-4688-aee4-783363723a40 None admin] PATH_INFO: `/v3/services/image` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:10.462914 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cf369d-312d-4688-aee4-783363723a40 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:10.464324 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cf369d-312d-4688-aee4-783363723a40 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:10.474088 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-c1cf369d-312d-4688-aee4-783363723a40 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 19 16:01:10.474088 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:01:10.474088 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 16:01:10.474088 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 16:01:10.474088 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 16:01:10.474088 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 16:01:10.474088 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 16:01:10.474088 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 16:01:10.474088 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 19 16:01:10.474088 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:01:10.474088 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 16:01:10.474088 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:01:10.474088 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:01:10.474088 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:01:10.474088 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:01:10.474088 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:01:10.474088 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:01:10.475302 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:01:10.475302 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:01:10.475302 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:01:10.475302 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:01:10.475302 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:01:10.475302 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:01:10.475302 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 16:01:10.475302 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 16:01:10.475302 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 16:01:10.475302 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 16:01:10.475302 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:01:10.475302 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:01:10.475302 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:01:10.475302 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:01:10.475302 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:01:10.475302 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:01:10.476146 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:01:10.476146 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:01:10.476146 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:01:10.476146 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:01:10.476146 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:01:10.476146 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:01:10.476146 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:01:10.476146 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:01:10.476146 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:01:10.476146 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:01:10.476146 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 16:01:10.476146 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 16:01:10.476146 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 19 16:01:10.476146 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:01:10.478709 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 505/1001] 10.222.0.57 () {56 vars in 1105 bytes} [Mon Sep 19 16:01:10 2022] GET /identity/v3/services/image => generated 86 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:01:10.481159 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d37937-ceae-4d97-a76c-d5bf793f23f8 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:10.528571 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cf369d-312d-4688-aee4-783363723a40 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:10.542034 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:10.586445 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d37937-ceae-4d97-a76c-d5bf793f23f8 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:10.621664 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5059a3-ae0c-4bd4-8e61-376a2a0c9cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:10.622271 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5059a3-ae0c-4bd4-8e61-376a2a0c9cda None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:10.622572 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5059a3-ae0c-4bd4-8e61-376a2a0c9cda None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:10.622841 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5059a3-ae0c-4bd4-8e61-376a2a0c9cda None admin] PATH_INFO: `/v3/services` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:10.623468 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5059a3-ae0c-4bd4-8e61-376a2a0c9cda None admin] RBAC: Adding query filter params (name=image) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:10.623767 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5059a3-ae0c-4bd4-8e61-376a2a0c9cda None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:10.625117 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5059a3-ae0c-4bd4-8e61-376a2a0c9cda None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:10.631232 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 497/1002] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:01:10 2022] GET /identity/v3/services?name=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:10.637139 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5059a3-ae0c-4bd4-8e61-376a2a0c9cda None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:10.647174 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cf369d-312d-4688-aee4-783363723a40 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:10.688939 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20166bfd-9739-41ea-b1d8-3b565118d2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:10.689500 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20166bfd-9739-41ea-b1d8-3b565118d2f6 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:10.689787 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20166bfd-9739-41ea-b1d8-3b565118d2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:10.690066 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20166bfd-9739-41ea-b1d8-3b565118d2f6 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:10.690696 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20166bfd-9739-41ea-b1d8-3b565118d2f6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:10.692072 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20166bfd-9739-41ea-b1d8-3b565118d2f6 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:10.702252 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-20166bfd-9739-41ea-b1d8-3b565118d2f6 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 19 16:01:10.702252 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:01:10.702252 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 16:01:10.702252 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 16:01:10.702252 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 16:01:10.702252 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 16:01:10.702252 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 16:01:10.702252 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 16:01:10.702252 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 19 16:01:10.702252 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:01:10.702252 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 16:01:10.702252 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:01:10.702252 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:01:10.702252 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:01:10.702252 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:01:10.702252 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:01:10.702252 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:01:10.708377 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:01:10.708377 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:01:10.708377 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:01:10.708377 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:01:10.708377 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:01:10.708377 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:01:10.708377 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 16:01:10.708377 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 16:01:10.708377 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 16:01:10.708377 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 16:01:10.708377 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:01:10.708377 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:01:10.708377 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:01:10.708377 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:01:10.708377 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:01:10.708377 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:01:10.709295 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:01:10.709295 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:01:10.709295 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:01:10.709295 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:01:10.709295 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:01:10.709295 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:01:10.709295 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:01:10.709295 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:01:10.709295 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:01:10.709295 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:01:10.709295 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 16:01:10.709295 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 16:01:10.709295 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 19 16:01:10.709295 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:01:10.709295 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 506/1003] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 19 16:01:10 2022] GET /identity/v3/services/compute => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:01:10.710093 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20166bfd-9739-41ea-b1d8-3b565118d2f6 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:10.749125 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:10.755079 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5059a3-ae0c-4bd4-8e61-376a2a0c9cda None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:10.785856 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:10.809918 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e52b361-14ed-4053-aac0-f1d39a1d6e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:10.810627 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e52b361-14ed-4053-aac0-f1d39a1d6e37 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:10.810906 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e52b361-14ed-4053-aac0-f1d39a1d6e37 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:10.811176 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e52b361-14ed-4053-aac0-f1d39a1d6e37 None admin] PATH_INFO: `/v3/services` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:10.811877 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e52b361-14ed-4053-aac0-f1d39a1d6e37 None admin] RBAC: Adding query filter params (type=image) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:10.812184 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e52b361-14ed-4053-aac0-f1d39a1d6e37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:10.813569 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e52b361-14ed-4053-aac0-f1d39a1d6e37 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:10.818344 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 498/1004] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:01:10 2022] GET /identity/v3/services?type=image => generated 122 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:01:10.834792 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20166bfd-9739-41ea-b1d8-3b565118d2f6 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:10.874223 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bf2bf0-3371-498a-bfbf-789522b5261e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:10.874884 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf2bf0-3371-498a-bfbf-789522b5261e None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:10.875158 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf2bf0-3371-498a-bfbf-789522b5261e None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:10.875427 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf2bf0-3371-498a-bfbf-789522b5261e None admin] PATH_INFO: `/v3/services` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:10.876068 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bf2bf0-3371-498a-bfbf-789522b5261e None admin] RBAC: Adding query filter params (name=compute) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:10.876373 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bf2bf0-3371-498a-bfbf-789522b5261e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:10.877781 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bf2bf0-3371-498a-bfbf-789522b5261e None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:10.883706 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 507/1005] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Sep 19 16:01:10 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:10.890047 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e52b361-14ed-4053-aac0-f1d39a1d6e37 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:11.008654 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e52b361-14ed-4053-aac0-f1d39a1d6e37 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:11.049228 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4076effa-7733-447f-a4d4-6448e7ea6daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:11.049841 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4076effa-7733-447f-a4d4-6448e7ea6daa None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:11.050122 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4076effa-7733-447f-a4d4-6448e7ea6daa None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:11.050416 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4076effa-7733-447f-a4d4-6448e7ea6daa None admin] PATH_INFO: `/v3/services` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:11.051050 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4076effa-7733-447f-a4d4-6448e7ea6daa None admin] RBAC: Adding query filter params (type=compute) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:11.051356 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4076effa-7733-447f-a4d4-6448e7ea6daa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:11.052725 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4076effa-7733-447f-a4d4-6448e7ea6daa None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:11.057883 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 499/1006] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Sep 19 16:01:10 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:11.064247 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bf2bf0-3371-498a-bfbf-789522b5261e None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:11.170762 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:11.174703 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bf2bf0-3371-498a-bfbf-789522b5261e None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:11.189756 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81820f30-ecd0-451f-8a00-6818ec2fa1b8 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:11.190279 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81820f30-ecd0-451f-8a00-6818ec2fa1b8 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:11.190603 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81820f30-ecd0-451f-8a00-6818ec2fa1b8 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:11.192454 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 500/1007] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:01:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 16:01:11.197454 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938b3814-167e-49fe-83cb-c4a1f1c34fea None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:11.197731 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938b3814-167e-49fe-83cb-c4a1f1c34fea None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:11.197997 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938b3814-167e-49fe-83cb-c4a1f1c34fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:11.212819 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e716eab-2c97-4bb6-a393-50d5706dfcda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:11.213433 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e716eab-2c97-4bb6-a393-50d5706dfcda None admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:11.213714 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e716eab-2c97-4bb6-a393-50d5706dfcda None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:11.214099 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e716eab-2c97-4bb6-a393-50d5706dfcda None admin] PATH_INFO: `/v3/endpoints` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:11.214761 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e716eab-2c97-4bb6-a393-50d5706dfcda None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:11.216117 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e716eab-2c97-4bb6-a393-50d5706dfcda None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:11.216513 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-938b3814-167e-49fe-83cb-c4a1f1c34fea None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:01:11.217690 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-938b3814-167e-49fe-83cb-c4a1f1c34fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:01:11.246399 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 508/1008] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:01:11 2022] POST /identity/v3/endpoints => generated 334 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 16:01:11.277454 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:11.314254 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 501/1009] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:01:11 2022] POST /identity/v3/auth/tokens => generated 1726 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 16:01:11.582856 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e716eab-2c97-4bb6-a393-50d5706dfcda None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:11.684804 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e716eab-2c97-4bb6-a393-50d5706dfcda None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:11.719512 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07da28f8-f48b-404b-8e1e-26343882fddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:11.720172 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da28f8-f48b-404b-8e1e-26343882fddc None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:11.720457 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da28f8-f48b-404b-8e1e-26343882fddc None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:11.720728 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da28f8-f48b-404b-8e1e-26343882fddc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:11.721326 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07da28f8-f48b-404b-8e1e-26343882fddc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:11.722768 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07da28f8-f48b-404b-8e1e-26343882fddc None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:11.732450 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-07da28f8-f48b-404b-8e1e-26343882fddc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 16:01:11.732450 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:01:11.732450 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 16:01:11.732450 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 16:01:11.732450 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 16:01:11.732450 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 16:01:11.732450 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 16:01:11.732450 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 16:01:11.732450 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 16:01:11.732450 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:01:11.732450 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 16:01:11.732450 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:01:11.732450 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:01:11.732450 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:01:11.732450 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:01:11.732450 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:01:11.732450 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:01:11.733504 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:01:11.733504 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:01:11.733504 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:01:11.733504 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:01:11.733504 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:01:11.733504 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:01:11.733504 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 16:01:11.733504 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 16:01:11.733504 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 16:01:11.733504 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 16:01:11.733504 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:01:11.733504 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:01:11.733504 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:01:11.733504 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:01:11.733504 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:01:11.733504 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:01:11.735202 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:01:11.735202 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:01:11.735202 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:01:11.735202 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:01:11.735202 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:01:11.735202 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:01:11.735202 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:01:11.735202 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:01:11.735202 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:01:11.735202 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:01:11.735202 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 16:01:11.735202 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 16:01:11.735202 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 16:01:11.735202 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:01:11.735202 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 509/1010] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Sep 19 16:01:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:01:11.739925 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938b3814-167e-49fe-83cb-c4a1f1c34fea None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:11.856877 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938b3814-167e-49fe-83cb-c4a1f1c34fea None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:11.891922 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279e5f67-fb28-44f8-91d3-7c26059c0520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:11.892536 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279e5f67-fb28-44f8-91d3-7c26059c0520 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:11.892817 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279e5f67-fb28-44f8-91d3-7c26059c0520 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:11.893093 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279e5f67-fb28-44f8-91d3-7c26059c0520 None admin] PATH_INFO: `/v3/services` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:11.893714 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279e5f67-fb28-44f8-91d3-7c26059c0520 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:11.894018 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279e5f67-fb28-44f8-91d3-7c26059c0520 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:11.895399 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279e5f67-fb28-44f8-91d3-7c26059c0520 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:11.905106 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 502/1011] 10.222.0.57 () {56 vars in 1122 bytes} [Mon Sep 19 16:01:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:11.906310 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07da28f8-f48b-404b-8e1e-26343882fddc None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:11.931501 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:12.017047 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07da28f8-f48b-404b-8e1e-26343882fddc None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:12.053483 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597dbb76-94d8-48bd-badd-9437af730afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:12.054104 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597dbb76-94d8-48bd-badd-9437af730afb None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:12.054395 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597dbb76-94d8-48bd-badd-9437af730afb None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:12.054823 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597dbb76-94d8-48bd-badd-9437af730afb None admin] PATH_INFO: `/v3/services` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:12.055462 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597dbb76-94d8-48bd-badd-9437af730afb None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:12.055770 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597dbb76-94d8-48bd-badd-9437af730afb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:12.057174 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597dbb76-94d8-48bd-badd-9437af730afb None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:12.063523 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 510/1012] 10.222.0.57 () {56 vars in 1122 bytes} [Mon Sep 19 16:01:11 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:12.069242 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279e5f67-fb28-44f8-91d3-7c26059c0520 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:12.111968 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:12.175615 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279e5f67-fb28-44f8-91d3-7c26059c0520 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:12.209901 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137ba486-229f-4d54-960b-40e911fb1a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:12.210543 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137ba486-229f-4d54-960b-40e911fb1a12 None admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:12.210829 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137ba486-229f-4d54-960b-40e911fb1a12 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:12.211103 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137ba486-229f-4d54-960b-40e911fb1a12 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:12.211778 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137ba486-229f-4d54-960b-40e911fb1a12 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:12.213128 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137ba486-229f-4d54-960b-40e911fb1a12 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:12.238502 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 503/1013] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:01:12 2022] POST /identity/v3/endpoints => generated 325 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 16:01:13.042995 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b7604-0e5a-46cf-9583-611e20da3a74 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:13.043513 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b7604-0e5a-46cf-9583-611e20da3a74 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:13.043849 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b7604-0e5a-46cf-9583-611e20da3a74 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:13.046137 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 511/1014] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:01:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:01:13.053380 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33b7fba-41d8-44b2-822b-fda16d382e77 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:13.053707 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33b7fba-41d8-44b2-822b-fda16d382e77 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:13.053947 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33b7fba-41d8-44b2-822b-fda16d382e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:13.074337 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-f33b7fba-41d8-44b2-822b-fda16d382e77 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:01:13.075573 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-f33b7fba-41d8-44b2-822b-fda16d382e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:01:13.165821 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 504/1015] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:01:13 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 16:01:13.455496 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5b7604-0e5a-46cf-9583-611e20da3a74 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:13.550716 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:13.570414 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5b7604-0e5a-46cf-9583-611e20da3a74 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:13.589048 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90593476-d9b4-4a12-903a-b513fe1a7829 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:13.589363 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90593476-d9b4-4a12-903a-b513fe1a7829 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:13.589640 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90593476-d9b4-4a12-903a-b513fe1a7829 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:13.591320 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 505/1016] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:01:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 16:01:13.596710 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a55c6d-27c9-466d-a7e8-3f0660186736 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:13.597038 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a55c6d-27c9-466d-a7e8-3f0660186736 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:13.597308 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a55c6d-27c9-466d-a7e8-3f0660186736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:13.617841 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb6092a-e9f6-42e0-9832-482873121a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:13.618446 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-17a55c6d-27c9-466d-a7e8-3f0660186736 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:01:13.618562 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb6092a-e9f6-42e0-9832-482873121a61 None admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:13.618780 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb6092a-e9f6-42e0-9832-482873121a61 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:13.619009 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb6092a-e9f6-42e0-9832-482873121a61 None admin] PATH_INFO: `/v3/services` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:13.619660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-17a55c6d-27c9-466d-a7e8-3f0660186736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:01:13.619766 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb6092a-e9f6-42e0-9832-482873121a61 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:13.621549 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb6092a-e9f6-42e0-9832-482873121a61 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:13.638804 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 512/1017] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Sep 19 16:01:13 2022] POST /identity/v3/services => generated 242 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 16:01:13.642724 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:13.723208 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 506/1018] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:01:13 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 16:01:14.008192 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb6092a-e9f6-42e0-9832-482873121a61 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:14.123940 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb6092a-e9f6-42e0-9832-482873121a61 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:14.169773 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b82f5ef-9d7b-40af-ab14-3de615a73cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:14.170459 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b82f5ef-9d7b-40af-ab14-3de615a73cc5 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:14.170743 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b82f5ef-9d7b-40af-ab14-3de615a73cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:14.171014 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b82f5ef-9d7b-40af-ab14-3de615a73cc5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:14.183404 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b82f5ef-9d7b-40af-ab14-3de615a73cc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 16:01:14.188347 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-4b82f5ef-9d7b-40af-ab14-3de615a73cc5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 16:01:14.188347 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:01:14.188347 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:01:14.188347 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:01:14.188347 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:01:14.188347 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:01:14.188347 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:01:14.188347 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:01:14.188347 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:01:14.188347 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:01:14.188347 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:01:14.188347 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:01:14.188347 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 16:01:14.188347 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 16:01:14.188347 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 16:01:14.188347 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 16:01:14.189301 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 16:01:14.189301 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 16:01:14.189301 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 16:01:14.189301 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 16:01:14.189301 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:01:14.189301 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:01:14.189301 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:01:14.189301 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:01:14.189301 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:01:14.189301 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:01:14.189301 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:01:14.189301 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:01:14.189301 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:01:14.189301 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:01:14.189301 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:01:14.189301 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:01:14.190150 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:01:14.190150 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:01:14.190150 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:01:14.190150 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:01:14.190150 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 16:01:14.190150 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 16:01:14.190150 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 16:01:14.190150 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 16:01:14.190150 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 16:01:14.190150 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 16:01:14.190150 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 16:01:14.190150 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:01:14.194942 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 513/1019] 10.222.0.57 () {56 vars in 1120 bytes} [Mon Sep 19 16:01:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:01:14.195678 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a55c6d-27c9-466d-a7e8-3f0660186736 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:14.310454 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a55c6d-27c9-466d-a7e8-3f0660186736 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:14.349469 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864c9475-f609-43c9-8f6f-43c7873e90ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:14.350130 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c9475-f609-43c9-8f6f-43c7873e90ab None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:14.350449 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c9475-f609-43c9-8f6f-43c7873e90ab None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:14.350728 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c9475-f609-43c9-8f6f-43c7873e90ab None admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:14.351403 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864c9475-f609-43c9-8f6f-43c7873e90ab None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:14.351788 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864c9475-f609-43c9-8f6f-43c7873e90ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:14.353192 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864c9475-f609-43c9-8f6f-43c7873e90ab None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:14.363276 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 507/1020] 10.222.0.57 () {56 vars in 1115 bytes} [Mon Sep 19 16:01:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:14.369016 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b82f5ef-9d7b-40af-ab14-3de615a73cc5 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:14.374211 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:14.501069 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b82f5ef-9d7b-40af-ab14-3de615a73cc5 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:14.547467 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9020a2fd-2f69-438a-b852-dbfb5f9e29a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:14.548134 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9020a2fd-2f69-438a-b852-dbfb5f9e29a2 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:14.548414 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9020a2fd-2f69-438a-b852-dbfb5f9e29a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:14.548688 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9020a2fd-2f69-438a-b852-dbfb5f9e29a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:14.560816 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9020a2fd-2f69-438a-b852-dbfb5f9e29a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:14.562231 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9020a2fd-2f69-438a-b852-dbfb5f9e29a2 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:14.580347 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-9020a2fd-2f69-438a-b852-dbfb5f9e29a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:01:14.580347 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:01:14.580347 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:01:14.580347 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:01:14.580347 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:01:14.580347 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:01:14.580347 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:01:14.580347 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:01:14.580347 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:01:14.580347 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:01:14.580347 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:01:14.580347 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:01:14.580347 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:01:14.580347 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:01:14.580347 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:01:14.580347 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:01:14.581532 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:01:14.581532 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:01:14.581532 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:01:14.581532 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:01:14.581532 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:01:14.581532 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:01:14.581532 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:01:14.581532 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:01:14.581532 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:01:14.581532 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:01:14.581532 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:01:14.581532 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:01:14.581532 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:01:14.581532 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:01:14.581532 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:01:14.581532 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:01:14.581532 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:01:14.582565 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:01:14.582565 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:01:14.582565 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:01:14.583823 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 514/1021] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:01:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:01:14.587844 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864c9475-f609-43c9-8f6f-43c7873e90ab None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:14.605915 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:14.703574 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faff269-fc0e-4eb3-9d2b-615e7ae9ffe7 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:14.703855 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faff269-fc0e-4eb3-9d2b-615e7ae9ffe7 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:14.704231 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faff269-fc0e-4eb3-9d2b-615e7ae9ffe7 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:14.705616 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 515/1022] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:01:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 16:01:14.709025 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864c9475-f609-43c9-8f6f-43c7873e90ab None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:14.711481 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc8d8a-1737-4745-9264-5a9917aef247 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:14.711775 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc8d8a-1737-4745-9264-5a9917aef247 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:14.712051 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc8d8a-1737-4745-9264-5a9917aef247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:14.732042 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-a7bc8d8a-1737-4745-9264-5a9917aef247 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:01:14.733280 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-a7bc8d8a-1737-4745-9264-5a9917aef247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:01:14.748425 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1cae16-0321-4466-a25c-4ebb06776def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:14.749054 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1cae16-0321-4466-a25c-4ebb06776def None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:14.749332 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1cae16-0321-4466-a25c-4ebb06776def None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:14.749607 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1cae16-0321-4466-a25c-4ebb06776def None admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:14.750269 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1cae16-0321-4466-a25c-4ebb06776def None admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:14.750617 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1cae16-0321-4466-a25c-4ebb06776def None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:14.752012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1cae16-0321-4466-a25c-4ebb06776def None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:14.771852 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 508/1023] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:01:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:14.773949 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1cae16-0321-4466-a25c-4ebb06776def None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:14.800799 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:14.819541 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 516/1024] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:01:14 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 16:01:14.859866 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:14.897363 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1cae16-0321-4466-a25c-4ebb06776def None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:14.934150 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc52d966-4de0-43ab-ab60-62703962413c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:14.934804 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc52d966-4de0-43ab-ab60-62703962413c None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:14.935448 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc52d966-4de0-43ab-ab60-62703962413c None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:14.935448 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc52d966-4de0-43ab-ab60-62703962413c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:14.944034 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc52d966-4de0-43ab-ab60-62703962413c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:14.945458 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc52d966-4de0-43ab-ab60-62703962413c None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:14.956833 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-cc52d966-4de0-43ab-ab60-62703962413c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 19 16:01:14.956833 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:01:14.956833 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:01:14.956833 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:01:14.956833 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:01:14.956833 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:01:14.956833 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:01:14.956833 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:01:14.956833 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:01:14.956833 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:01:14.956833 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:01:14.956833 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:01:14.956833 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 16:01:14.956833 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 16:01:14.956833 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 16:01:14.956833 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 16:01:14.957807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:01:14.957807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:01:14.957807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 16:01:14.957807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:01:14.957807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 16:01:14.957807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:01:14.957807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:01:14.957807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:01:14.957807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:01:14.957807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:01:14.957807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:01:14.957807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:01:14.957807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:01:14.957807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:01:14.957807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:01:14.957807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:01:14.957807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:01:14.957807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:01:14.963788 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:01:14.963788 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:01:14.963788 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 16:01:14.963788 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 16:01:14.963788 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 16:01:14.963788 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 16:01:14.963788 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 16:01:14.963788 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 16:01:14.963788 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 19 16:01:14.963788 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:01:14.963788 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 509/1025] 10.222.0.57 () {56 vars in 1093 bytes} [Mon Sep 19 16:01:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:01:14.969267 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bc8d8a-1737-4745-9264-5a9917aef247 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:15.073358 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bc8d8a-1737-4745-9264-5a9917aef247 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:15.093239 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc52d966-4de0-43ab-ab60-62703962413c None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:15.110703 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1068e7-adb6-4087-8f9a-2314a8115acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:15.111450 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1068e7-adb6-4087-8f9a-2314a8115acf None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:15.111824 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1068e7-adb6-4087-8f9a-2314a8115acf None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:15.112201 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1068e7-adb6-4087-8f9a-2314a8115acf None admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:15.113016 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1068e7-adb6-4087-8f9a-2314a8115acf None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:15.113417 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1068e7-adb6-4087-8f9a-2314a8115acf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:15.114975 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1068e7-adb6-4087-8f9a-2314a8115acf None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:15.132830 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 517/1026] 10.222.0.57 () {56 vars in 1133 bytes} [Mon Sep 19 16:01:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:15.138502 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1068e7-adb6-4087-8f9a-2314a8115acf None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:15.189680 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:15.202484 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc52d966-4de0-43ab-ab60-62703962413c None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:15.233719 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:15.239393 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d16a5a-8260-4107-82a9-d90000d466c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:15.240037 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d16a5a-8260-4107-82a9-d90000d466c7 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:15.240297 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d16a5a-8260-4107-82a9-d90000d466c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:15.240567 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d16a5a-8260-4107-82a9-d90000d466c7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:15.241656 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d16a5a-8260-4107-82a9-d90000d466c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:15.243101 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d16a5a-8260-4107-82a9-d90000d466c7 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:15.244649 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 510/1027] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:01:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:01:15.247861 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1068e7-adb6-4087-8f9a-2314a8115acf None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:15.250423 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d16a5a-8260-4107-82a9-d90000d466c7 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:15.284222 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e31b99-022b-462b-bbfa-e2ca986d53f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:15.284803 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e31b99-022b-462b-bbfa-e2ca986d53f5 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:15.285080 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e31b99-022b-462b-bbfa-e2ca986d53f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:15.285348 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e31b99-022b-462b-bbfa-e2ca986d53f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:15.358875 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e31b99-022b-462b-bbfa-e2ca986d53f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:15.359482 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d16a5a-8260-4107-82a9-d90000d466c7 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:15.360241 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e31b99-022b-462b-bbfa-e2ca986d53f5 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:15.394769 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b79ba0-654c-4733-ae12-068675aa6468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:15.395385 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b79ba0-654c-4733-ae12-068675aa6468 None admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:15.395667 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b79ba0-654c-4733-ae12-068675aa6468 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:15.395938 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b79ba0-654c-4733-ae12-068675aa6468 None admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:15.396588 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b79ba0-654c-4733-ae12-068675aa6468 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:15.398003 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b79ba0-654c-4733-ae12-068675aa6468 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:15.405942 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:15.432557 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 511/1028] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Sep 19 16:01:15 2022] POST /identity/v3/projects => generated 311 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 16:01:15.492828 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 518/1029] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 19 16:01:15 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 19 16:01:15.494824 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b79ba0-654c-4733-ae12-068675aa6468 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:15.555209 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:15.609331 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b79ba0-654c-4733-ae12-068675aa6468 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:15.648471 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6ad501-1331-4e7f-91b0-4f4802a150a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:15.649103 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6ad501-1331-4e7f-91b0-4f4802a150a2 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:15.649378 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6ad501-1331-4e7f-91b0-4f4802a150a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:15.649645 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6ad501-1331-4e7f-91b0-4f4802a150a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:15.660548 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6ad501-1331-4e7f-91b0-4f4802a150a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:15.661997 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6ad501-1331-4e7f-91b0-4f4802a150a2 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:15.680821 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-cf6ad501-1331-4e7f-91b0-4f4802a150a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:01:15.680821 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:01:15.680821 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:01:15.680821 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:01:15.680821 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:01:15.680821 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:01:15.680821 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:01:15.680821 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:01:15.680821 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:01:15.680821 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:01:15.680821 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:01:15.680821 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:01:15.680821 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:01:15.680821 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:01:15.680821 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:01:15.680821 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:01:15.681955 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:01:15.681955 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:01:15.681955 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:01:15.681955 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:01:15.681955 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:01:15.681955 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:01:15.681955 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:01:15.681955 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:01:15.681955 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:01:15.681955 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:01:15.681955 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:01:15.681955 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:01:15.681955 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:01:15.681955 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:01:15.681955 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:01:15.681955 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:01:15.681955 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:01:15.682943 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:01:15.682943 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:01:15.682943 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:01:15.682943 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 512/1030] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:01:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:01:15.689608 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e31b99-022b-462b-bbfa-e2ca986d53f5 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:15.803859 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e31b99-022b-462b-bbfa-e2ca986d53f5 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:15.840336 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6d4292-82ef-4090-b35d-9ec5852ec9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:15.840954 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d4292-82ef-4090-b35d-9ec5852ec9ac None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:15.841292 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d4292-82ef-4090-b35d-9ec5852ec9ac None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:15.841571 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d4292-82ef-4090-b35d-9ec5852ec9ac None admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:15.842221 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6d4292-82ef-4090-b35d-9ec5852ec9ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:15.842557 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6d4292-82ef-4090-b35d-9ec5852ec9ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:15.843926 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6d4292-82ef-4090-b35d-9ec5852ec9ac None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:15.855034 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7fc199-8e5b-414b-8819-c5677cb44aef None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:15.855317 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7fc199-8e5b-414b-8819-c5677cb44aef None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:15.855584 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7fc199-8e5b-414b-8819-c5677cb44aef None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:15.856104 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 519/1031] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:01:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:15.856994 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 513/1032] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:01:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 16:01:15.861456 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6d4292-82ef-4090-b35d-9ec5852ec9ac None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:15.861910 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef180ff-a7d8-45b4-8ab2-a1faf6cac26a None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:15.862199 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef180ff-a7d8-45b4-8ab2-a1faf6cac26a None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:15.862522 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef180ff-a7d8-45b4-8ab2-a1faf6cac26a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:15.866283 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:15.885088 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-3ef180ff-a7d8-45b4-8ab2-a1faf6cac26a None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:01:15.887338 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-3ef180ff-a7d8-45b4-8ab2-a1faf6cac26a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:01:15.969174 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6d4292-82ef-4090-b35d-9ec5852ec9ac None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:15.969813 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 514/1033] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:01:15 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 16:01:16.012027 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441686a7-5b7c-4ca5-909c-0fa4da29a540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:16.012692 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441686a7-5b7c-4ca5-909c-0fa4da29a540 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:16.012972 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441686a7-5b7c-4ca5-909c-0fa4da29a540 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:16.013245 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441686a7-5b7c-4ca5-909c-0fa4da29a540 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:16.020420 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441686a7-5b7c-4ca5-909c-0fa4da29a540 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:16.021859 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441686a7-5b7c-4ca5-909c-0fa4da29a540 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:16.034193 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-441686a7-5b7c-4ca5-909c-0fa4da29a540 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:01:16.034193 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:01:16.034193 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:01:16.034193 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:01:16.034193 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:01:16.034193 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:01:16.034193 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:01:16.034193 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:01:16.034193 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:01:16.034193 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:01:16.034193 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:01:16.034193 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:01:16.034193 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 16:01:16.034193 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 16:01:16.034193 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 16:01:16.041174 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 16:01:16.041174 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:01:16.041174 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:01:16.041174 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:01:16.041174 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:01:16.041174 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:01:16.041174 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:01:16.041174 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:01:16.041174 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:01:16.041174 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:01:16.041174 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:01:16.041174 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:01:16.041174 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:01:16.041174 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:01:16.041174 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:01:16.041174 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:01:16.041174 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:01:16.042326 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 16:01:16.042326 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 16:01:16.042326 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 16:01:16.042326 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 16:01:16.042326 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 16:01:16.042326 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 16:01:16.042326 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:01:16.042326 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:01:16.042326 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 520/1034] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 19 16:01:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:01:16.042850 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef180ff-a7d8-45b4-8ab2-a1faf6cac26a None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:16.089304 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:16.159443 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef180ff-a7d8-45b4-8ab2-a1faf6cac26a None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:16.193331 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f52a99-5bf7-43db-85d8-75ce22430bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:16.193944 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f52a99-5bf7-43db-85d8-75ce22430bf5 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:16.194223 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f52a99-5bf7-43db-85d8-75ce22430bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:16.194529 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f52a99-5bf7-43db-85d8-75ce22430bf5 None admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:16.195221 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f52a99-5bf7-43db-85d8-75ce22430bf5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:16.195523 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f52a99-5bf7-43db-85d8-75ce22430bf5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:16.196942 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f52a99-5bf7-43db-85d8-75ce22430bf5 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:16.209212 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 515/1035] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 19 16:01:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:16.216263 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441686a7-5b7c-4ca5-909c-0fa4da29a540 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:16.279758 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:16.284517 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f52a99-5bf7-43db-85d8-75ce22430bf5 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:16.326315 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441686a7-5b7c-4ca5-909c-0fa4da29a540 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:16.379858 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa783b9-132a-4ddd-9b84-352dacebe5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:16.380481 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa783b9-132a-4ddd-9b84-352dacebe5c4 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:16.380781 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa783b9-132a-4ddd-9b84-352dacebe5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:16.381037 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa783b9-132a-4ddd-9b84-352dacebe5c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:16.381831 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa783b9-132a-4ddd-9b84-352dacebe5c4 None admin] RBAC: Adding query filter params (role.id=c30e151440d44fd18e607056b4d41ec7, scope.project.id=be03e5300b3343169f35c643c93d1e06, user.id=1ecd01cc72834b72b5ff2c4b47249737) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:16.382141 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa783b9-132a-4ddd-9b84-352dacebe5c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:16.383622 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa783b9-132a-4ddd-9b84-352dacebe5c4 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:16.394565 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 521/1036] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 19 16:01:16 2022] GET /identity/v3/role_assignments?user.id=1ecd01cc72834b72b5ff2c4b47249737&scope.project.id=be03e5300b3343169f35c643c93d1e06&role.id=c30e151440d44fd18e607056b4d41ec7 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:16.398897 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:16.408682 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f52a99-5bf7-43db-85d8-75ce22430bf5 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:16.455363 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b92c68b-0016-4976-a3d9-8d01e0842f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:16.456024 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b92c68b-0016-4976-a3d9-8d01e0842f9a None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:16.456303 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b92c68b-0016-4976-a3d9-8d01e0842f9a None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:16.456579 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b92c68b-0016-4976-a3d9-8d01e0842f9a None admin] PATH_INFO: `/v3/services/image` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:16.457173 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b92c68b-0016-4976-a3d9-8d01e0842f9a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:16.458606 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b92c68b-0016-4976-a3d9-8d01e0842f9a None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:16.468956 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-1b92c68b-0016-4976-a3d9-8d01e0842f9a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 19 16:01:16.468956 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:01:16.468956 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 16:01:16.468956 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 16:01:16.468956 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 16:01:16.468956 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 16:01:16.468956 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 16:01:16.468956 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 16:01:16.468956 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 19 16:01:16.468956 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:01:16.468956 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 16:01:16.468956 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:01:16.468956 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:01:16.468956 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:01:16.468956 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:01:16.468956 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:01:16.468956 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:01:16.470133 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:01:16.470133 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:01:16.470133 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:01:16.470133 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:01:16.470133 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:01:16.470133 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:01:16.470133 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 16:01:16.470133 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 16:01:16.470133 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 16:01:16.470133 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 16:01:16.470133 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:01:16.470133 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:01:16.470133 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:01:16.470133 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:01:16.470133 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:01:16.470133 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:01:16.475607 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:01:16.475607 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:01:16.475607 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:01:16.475607 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:01:16.475607 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:01:16.475607 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:01:16.475607 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:01:16.475607 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:01:16.475607 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:01:16.475607 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:01:16.475607 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 16:01:16.475607 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 16:01:16.475607 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 19 16:01:16.475607 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:01:16.475607 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 516/1037] 10.222.0.57 () {56 vars in 1105 bytes} [Mon Sep 19 16:01:16 2022] GET /identity/v3/services/image => generated 86 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 16:01:16.476987 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa783b9-132a-4ddd-9b84-352dacebe5c4 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:16.595874 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa783b9-132a-4ddd-9b84-352dacebe5c4 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:16.623981 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:16.632596 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215d416e-dc27-48bf-83ad-6238a6905419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:16.633206 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215d416e-dc27-48bf-83ad-6238a6905419 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:16.633482 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215d416e-dc27-48bf-83ad-6238a6905419 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:16.633752 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215d416e-dc27-48bf-83ad-6238a6905419 None admin] PATH_INFO: `/v3/services` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:16.634388 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215d416e-dc27-48bf-83ad-6238a6905419 None admin] RBAC: Adding query filter params (name=image) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:16.634700 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215d416e-dc27-48bf-83ad-6238a6905419 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:16.636085 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215d416e-dc27-48bf-83ad-6238a6905419 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:16.646440 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 522/1038] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:01:16 2022] GET /identity/v3/services?name=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:16.647596 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b92c68b-0016-4976-a3d9-8d01e0842f9a None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:16.782746 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b92c68b-0016-4976-a3d9-8d01e0842f9a None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:16.817743 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17d32be-a144-4867-abbc-30da8970619a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:16.818356 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d32be-a144-4867-abbc-30da8970619a None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:16.818698 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d32be-a144-4867-abbc-30da8970619a None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:16.818978 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d32be-a144-4867-abbc-30da8970619a None admin] PATH_INFO: `/v3/services` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:16.819607 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17d32be-a144-4867-abbc-30da8970619a None admin] RBAC: Adding query filter params (type=image) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:16.819912 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17d32be-a144-4867-abbc-30da8970619a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:16.821272 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17d32be-a144-4867-abbc-30da8970619a None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:16.827057 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 517/1039] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:01:16 2022] GET /identity/v3/services?type=image => generated 350 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:01:16.832993 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215d416e-dc27-48bf-83ad-6238a6905419 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:16.938779 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215d416e-dc27-48bf-83ad-6238a6905419 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:16.978747 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b0ba15-c425-4729-b1be-5f81f1a4684d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:16.979378 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b0ba15-c425-4729-b1be-5f81f1a4684d None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:16.979669 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b0ba15-c425-4729-b1be-5f81f1a4684d None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:16.980048 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b0ba15-c425-4729-b1be-5f81f1a4684d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:16.980760 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b0ba15-c425-4729-b1be-5f81f1a4684d None admin] RBAC: Adding query filter params (interface=public, service_id=267b46fcd9cb4ed0ae5a6a3b3fa8c3dc, region_id=RegionOne) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:16.981085 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b0ba15-c425-4729-b1be-5f81f1a4684d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:16.982777 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b0ba15-c425-4729-b1be-5f81f1a4684d None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:16.996640 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 523/1040] 10.222.0.57 () {56 vars in 1251 bytes} [Mon Sep 19 16:01:16 2022] GET /identity/v3/endpoints?service_id=267b46fcd9cb4ed0ae5a6a3b3fa8c3dc&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:16.998427 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17d32be-a144-4867-abbc-30da8970619a None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:17.028560 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:17.111003 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17d32be-a144-4867-abbc-30da8970619a None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:17.147165 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ba41c4-6c2a-4f5f-94ea-0790f2307010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:17.147774 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba41c4-6c2a-4f5f-94ea-0790f2307010 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:17.148051 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba41c4-6c2a-4f5f-94ea-0790f2307010 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:17.148322 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba41c4-6c2a-4f5f-94ea-0790f2307010 None admin] PATH_INFO: `/v3/services` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:17.148915 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ba41c4-6c2a-4f5f-94ea-0790f2307010 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:17.150259 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ba41c4-6c2a-4f5f-94ea-0790f2307010 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:17.157091 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 518/1041] 10.222.0.57 () {56 vars in 1087 bytes} [Mon Sep 19 16:01:16 2022] GET /identity/v3/services => generated 1957 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:01:17.702703 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581b071f-a27f-44fe-b4dc-09616b3a0230 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:17.703254 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581b071f-a27f-44fe-b4dc-09616b3a0230 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:17.703254 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581b071f-a27f-44fe-b4dc-09616b3a0230 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:17.704608 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 524/1042] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:01:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:01:17.709977 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf786c0-a135-4fd9-9d92-aa5f09754c95 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:17.710522 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf786c0-a135-4fd9-9d92-aa5f09754c95 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:17.710629 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf786c0-a135-4fd9-9d92-aa5f09754c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:17.729022 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-9bf786c0-a135-4fd9-9d92-aa5f09754c95 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:01:17.730229 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-9bf786c0-a135-4fd9-9d92-aa5f09754c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:01:17.803792 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 519/1043] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:01:17 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 16:01:18.077052 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581b071f-a27f-44fe-b4dc-09616b3a0230 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:18.140519 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:18.189346 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581b071f-a27f-44fe-b4dc-09616b3a0230 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:18.224346 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374c0864-743a-4f40-bc17-c8a940327556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:18.225037 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374c0864-743a-4f40-bc17-c8a940327556 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:18.225311 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374c0864-743a-4f40-bc17-c8a940327556 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:18.225577 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374c0864-743a-4f40-bc17-c8a940327556 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:18.226784 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374c0864-743a-4f40-bc17-c8a940327556 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:18.228190 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374c0864-743a-4f40-bc17-c8a940327556 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:18.230411 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 525/1044] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:01:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:18.235724 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf786c0-a135-4fd9-9d92-aa5f09754c95 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:18.253857 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:18.341352 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf786c0-a135-4fd9-9d92-aa5f09754c95 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:18.374820 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d94074a-a782-410f-9bb4-fc2ddf8d072a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:18.375433 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d94074a-a782-410f-9bb4-fc2ddf8d072a None admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:18.375713 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d94074a-a782-410f-9bb4-fc2ddf8d072a None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:18.375985 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d94074a-a782-410f-9bb4-fc2ddf8d072a None admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:18.376640 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d94074a-a782-410f-9bb4-fc2ddf8d072a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:18.378184 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d94074a-a782-410f-9bb4-fc2ddf8d072a None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:18.402817 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 520/1045] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Sep 19 16:01:18 2022] POST /identity/v3/users => generated 285 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 16:01:18.592234 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bada2f28-b29a-4851-af43-fb049d7c69cc None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:18.592426 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bada2f28-b29a-4851-af43-fb049d7c69cc None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:18.592698 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bada2f28-b29a-4851-af43-fb049d7c69cc None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:18.594171 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 526/1046] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:01:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:01:18.599302 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ae60a-ff61-4242-838e-e0430e1bc6b8 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:18.599583 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ae60a-ff61-4242-838e-e0430e1bc6b8 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:18.599858 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ae60a-ff61-4242-838e-e0430e1bc6b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:18.618845 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-050ae60a-ff61-4242-838e-e0430e1bc6b8 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:01:18.620020 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-050ae60a-ff61-4242-838e-e0430e1bc6b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:01:18.693493 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 521/1047] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:01:18 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 16:01:18.967994 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bada2f28-b29a-4851-af43-fb049d7c69cc None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:19.085042 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bada2f28-b29a-4851-af43-fb049d7c69cc None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:19.109858 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:19.121809 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f2a384-6ea8-4fef-b216-840b5c4781a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:19.122481 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f2a384-6ea8-4fef-b216-840b5c4781a4 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:19.122761 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f2a384-6ea8-4fef-b216-840b5c4781a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:19.123031 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f2a384-6ea8-4fef-b216-840b5c4781a4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:19.133682 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f2a384-6ea8-4fef-b216-840b5c4781a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 16:01:19.139313 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-76f2a384-6ea8-4fef-b216-840b5c4781a4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 16:01:19.139313 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:01:19.139313 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:01:19.139313 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:01:19.139313 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:01:19.139313 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:01:19.139313 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:01:19.139313 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:01:19.139313 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:01:19.139313 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:01:19.139313 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:01:19.139313 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:01:19.139313 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 16:01:19.139313 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 16:01:19.139313 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 16:01:19.139313 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 16:01:19.140406 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 16:01:19.140406 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 16:01:19.140406 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 16:01:19.140406 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 16:01:19.140406 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:01:19.140406 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:01:19.140406 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:01:19.140406 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:01:19.140406 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:01:19.140406 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:01:19.140406 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:01:19.140406 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:01:19.140406 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:01:19.140406 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:01:19.140406 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:01:19.140406 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:01:19.141310 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:01:19.141310 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:01:19.141310 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:01:19.141310 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:01:19.141310 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 16:01:19.141310 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 16:01:19.141310 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 16:01:19.141310 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 16:01:19.141310 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 16:01:19.141310 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 16:01:19.141310 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 16:01:19.141310 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:01:19.145764 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 527/1048] 10.222.0.57 () {56 vars in 1120 bytes} [Mon Sep 19 16:01:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:01:19.147055 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050ae60a-ff61-4242-838e-e0430e1bc6b8 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:19.149618 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:19.255262 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050ae60a-ff61-4242-838e-e0430e1bc6b8 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:19.293784 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa3395d-54cc-4fee-99e4-085cacc1fcf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:19.294404 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3395d-54cc-4fee-99e4-085cacc1fcf7 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:19.294679 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3395d-54cc-4fee-99e4-085cacc1fcf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:19.294949 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3395d-54cc-4fee-99e4-085cacc1fcf7 None admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:19.295608 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa3395d-54cc-4fee-99e4-085cacc1fcf7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:19.295915 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa3395d-54cc-4fee-99e4-085cacc1fcf7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:19.297263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa3395d-54cc-4fee-99e4-085cacc1fcf7 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:19.310419 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 522/1049] 10.222.0.57 () {56 vars in 1115 bytes} [Mon Sep 19 16:01:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:19.312527 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f2a384-6ea8-4fef-b216-840b5c4781a4 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:19.375310 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a379587d-ef11-44d1-b5f5-d8899c774205 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:19.375853 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a379587d-ef11-44d1-b5f5-d8899c774205 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:19.376131 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a379587d-ef11-44d1-b5f5-d8899c774205 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:19.377977 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 523/1050] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:01:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 16:01:19.383318 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c28fd-3315-49aa-8722-9c477f2adfa2 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:19.383590 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c28fd-3315-49aa-8722-9c477f2adfa2 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:19.383836 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c28fd-3315-49aa-8722-9c477f2adfa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:19.403690 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-6f2c28fd-3315-49aa-8722-9c477f2adfa2 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:01:19.404951 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-6f2c28fd-3315-49aa-8722-9c477f2adfa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:01:19.422440 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f2a384-6ea8-4fef-b216-840b5c4781a4 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:19.459648 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b48e45-aa3b-4649-8055-980e514706da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:19.460289 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b48e45-aa3b-4649-8055-980e514706da None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:19.460568 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b48e45-aa3b-4649-8055-980e514706da None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:19.460837 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b48e45-aa3b-4649-8055-980e514706da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:19.472022 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b48e45-aa3b-4649-8055-980e514706da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:19.473380 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b48e45-aa3b-4649-8055-980e514706da None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:19.473657 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:19.483627 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 524/1051] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:01:19 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 16:01:19.490603 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-67b48e45-aa3b-4649-8055-980e514706da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:01:19.490603 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:01:19.490603 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:01:19.490603 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:01:19.490603 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:01:19.490603 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:01:19.490603 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:01:19.490603 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:01:19.490603 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:01:19.490603 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:01:19.490603 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:01:19.490603 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:01:19.490603 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:01:19.490603 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:01:19.490603 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:01:19.490603 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:01:19.491569 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:01:19.491569 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:01:19.491569 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:01:19.491569 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:01:19.491569 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:01:19.491569 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:01:19.491569 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:01:19.491569 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:01:19.491569 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:01:19.491569 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:01:19.491569 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:01:19.491569 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:01:19.491569 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:01:19.491569 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:01:19.491569 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:01:19.491569 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:01:19.491569 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:01:19.492504 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:01:19.492504 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:01:19.492504 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:01:19.494713 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 528/1052] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:01:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:01:19.497309 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2c28fd-3315-49aa-8722-9c477f2adfa2 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:19.598389 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2c28fd-3315-49aa-8722-9c477f2adfa2 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:19.632754 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a8646e-d41f-4dba-91a8-dd9ffcc9c7bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:19.633357 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a8646e-d41f-4dba-91a8-dd9ffcc9c7bb None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:19.633641 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a8646e-d41f-4dba-91a8-dd9ffcc9c7bb None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:19.633914 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a8646e-d41f-4dba-91a8-dd9ffcc9c7bb None admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:19.634590 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a8646e-d41f-4dba-91a8-dd9ffcc9c7bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:19.634885 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a8646e-d41f-4dba-91a8-dd9ffcc9c7bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:19.636780 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a8646e-d41f-4dba-91a8-dd9ffcc9c7bb None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:19.652480 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 525/1053] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:01:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:19.653889 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b48e45-aa3b-4649-8055-980e514706da None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:19.691380 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:19.752854 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a8646e-d41f-4dba-91a8-dd9ffcc9c7bb None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:19.759846 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b48e45-aa3b-4649-8055-980e514706da None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:19.800461 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1403b1a9-ec70-451d-bcec-938c65a4be94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:19.801083 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1403b1a9-ec70-451d-bcec-938c65a4be94 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:19.801358 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1403b1a9-ec70-451d-bcec-938c65a4be94 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:19.801626 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1403b1a9-ec70-451d-bcec-938c65a4be94 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:19.808960 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1403b1a9-ec70-451d-bcec-938c65a4be94 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:19.810415 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1403b1a9-ec70-451d-bcec-938c65a4be94 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:19.820422 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:19.822521 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-1403b1a9-ec70-451d-bcec-938c65a4be94 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 19 16:01:19.822521 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:01:19.822521 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:01:19.822521 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:01:19.822521 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:01:19.822521 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:01:19.822521 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:01:19.822521 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:01:19.822521 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:01:19.822521 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:01:19.822521 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:01:19.822521 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:01:19.822521 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 16:01:19.822521 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 16:01:19.822521 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 16:01:19.822521 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 16:01:19.824240 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:01:19.824240 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:01:19.824240 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 16:01:19.824240 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:01:19.824240 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 16:01:19.824240 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:01:19.824240 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:01:19.824240 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:01:19.824240 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:01:19.824240 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:01:19.824240 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:01:19.824240 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:01:19.824240 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:01:19.824240 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:01:19.824240 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:01:19.824240 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:01:19.824240 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:01:19.824240 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:01:19.825857 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:01:19.825857 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:01:19.825857 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 16:01:19.825857 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 16:01:19.825857 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 16:01:19.825857 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 16:01:19.825857 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 16:01:19.825857 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 16:01:19.825857 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 19 16:01:19.825857 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:01:19.825857 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 529/1054] 10.222.0.57 () {56 vars in 1093 bytes} [Mon Sep 19 16:01:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:01:19.829999 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1403b1a9-ec70-451d-bcec-938c65a4be94 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:19.876180 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a8646e-d41f-4dba-91a8-dd9ffcc9c7bb None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:19.915540 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2737ff4-567b-4ee6-b481-c664d76f348d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:19.916345 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2737ff4-567b-4ee6-b481-c664d76f348d None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:19.916678 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2737ff4-567b-4ee6-b481-c664d76f348d None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:19.917402 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2737ff4-567b-4ee6-b481-c664d76f348d None admin] PATH_INFO: `/v3/services/image` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:19.917936 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2737ff4-567b-4ee6-b481-c664d76f348d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:19.919756 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2737ff4-567b-4ee6-b481-c664d76f348d None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:19.931768 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-b2737ff4-567b-4ee6-b481-c664d76f348d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 19 16:01:19.931768 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:01:19.931768 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 16:01:19.931768 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 16:01:19.931768 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 16:01:19.931768 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 16:01:19.931768 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 16:01:19.931768 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 16:01:19.931768 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 19 16:01:19.931768 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:01:19.931768 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 16:01:19.931768 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:01:19.931768 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:01:19.931768 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:01:19.931768 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:01:19.931768 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:01:19.931768 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:01:19.933211 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:01:19.933211 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:01:19.933211 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:01:19.933211 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:01:19.933211 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:01:19.933211 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:01:19.933211 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 16:01:19.933211 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 16:01:19.933211 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 16:01:19.933211 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 16:01:19.933211 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:01:19.933211 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:01:19.933211 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:01:19.933211 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:01:19.933211 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:01:19.933211 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:01:19.934280 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:01:19.934280 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:01:19.934280 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:01:19.934280 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:01:19.934280 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:01:19.934280 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:01:19.934280 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:01:19.934280 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:01:19.934280 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:01:19.934280 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:01:19.934280 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 16:01:19.934280 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 16:01:19.934280 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 19 16:01:19.934280 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:01:19.938325 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 526/1055] 10.222.0.57 () {56 vars in 1105 bytes} [Mon Sep 19 16:01:19 2022] GET /identity/v3/services/image => generated 86 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 16:01:19.939808 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2737ff4-567b-4ee6-b481-c664d76f348d None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:19.944263 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:19.951317 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1403b1a9-ec70-451d-bcec-938c65a4be94 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:19.988148 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714150dc-cf72-4062-a262-e38a6211aeb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:19.988824 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714150dc-cf72-4062-a262-e38a6211aeb8 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:19.988994 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714150dc-cf72-4062-a262-e38a6211aeb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:19.989262 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714150dc-cf72-4062-a262-e38a6211aeb8 None admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:19.989983 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714150dc-cf72-4062-a262-e38a6211aeb8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:19.990287 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714150dc-cf72-4062-a262-e38a6211aeb8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:19.991740 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714150dc-cf72-4062-a262-e38a6211aeb8 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:20.014336 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 530/1056] 10.222.0.57 () {56 vars in 1133 bytes} [Mon Sep 19 16:01:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:20.019724 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714150dc-cf72-4062-a262-e38a6211aeb8 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:20.061334 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2737ff4-567b-4ee6-b481-c664d76f348d None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:20.079320 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:20.098206 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079fabc5-604a-40ab-b2c3-2ceb71be2d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:20.098843 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079fabc5-604a-40ab-b2c3-2ceb71be2d4d None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:20.099119 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079fabc5-604a-40ab-b2c3-2ceb71be2d4d None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:20.099390 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079fabc5-604a-40ab-b2c3-2ceb71be2d4d None admin] PATH_INFO: `/v3/services` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:20.100013 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079fabc5-604a-40ab-b2c3-2ceb71be2d4d None admin] RBAC: Adding query filter params (name=image) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:20.100316 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079fabc5-604a-40ab-b2c3-2ceb71be2d4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:20.101662 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079fabc5-604a-40ab-b2c3-2ceb71be2d4d None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:20.110413 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 527/1057] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:01:19 2022] GET /identity/v3/services?name=image => generated 122 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:01:20.113281 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079fabc5-604a-40ab-b2c3-2ceb71be2d4d None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:20.130314 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714150dc-cf72-4062-a262-e38a6211aeb8 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:20.166233 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778b0c48-e6b4-4b83-8b72-b2db06220053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:20.167159 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778b0c48-e6b4-4b83-8b72-b2db06220053 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:20.168062 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778b0c48-e6b4-4b83-8b72-b2db06220053 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:20.168851 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778b0c48-e6b4-4b83-8b72-b2db06220053 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:20.180107 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778b0c48-e6b4-4b83-8b72-b2db06220053 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:20.181626 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778b0c48-e6b4-4b83-8b72-b2db06220053 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:20.196762 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-778b0c48-e6b4-4b83-8b72-b2db06220053 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:01:20.196762 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:01:20.196762 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:01:20.196762 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:01:20.196762 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:01:20.196762 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:01:20.196762 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:01:20.196762 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:01:20.196762 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:01:20.196762 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:01:20.196762 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:01:20.196762 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:01:20.196762 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:01:20.196762 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:01:20.196762 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:01:20.196762 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:01:20.197775 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:01:20.197775 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:01:20.197775 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:01:20.197775 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:01:20.197775 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:01:20.197775 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:01:20.197775 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:01:20.197775 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:01:20.197775 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:01:20.197775 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:01:20.197775 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:01:20.197775 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:01:20.197775 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:01:20.197775 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:01:20.197775 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:01:20.197775 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:01:20.197775 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:01:20.200090 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:01:20.200090 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:01:20.200090 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:01:20.200090 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 531/1058] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:01:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 16:01:20.203962 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778b0c48-e6b4-4b83-8b72-b2db06220053 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:20.213601 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:20.223141 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079fabc5-604a-40ab-b2c3-2ceb71be2d4d None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:20.259547 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a75b2d-75fc-46ca-8a1a-9266f878793e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:20.260154 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a75b2d-75fc-46ca-8a1a-9266f878793e None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:20.260429 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a75b2d-75fc-46ca-8a1a-9266f878793e None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:20.260698 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a75b2d-75fc-46ca-8a1a-9266f878793e None admin] PATH_INFO: `/v3/services` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:20.262009 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a75b2d-75fc-46ca-8a1a-9266f878793e None admin] RBAC: Adding query filter params (type=image) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:20.262181 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a75b2d-75fc-46ca-8a1a-9266f878793e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:20.263687 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a75b2d-75fc-46ca-8a1a-9266f878793e None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:20.268977 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 528/1059] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:01:20 2022] GET /identity/v3/services?type=image => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:01:20.274558 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a75b2d-75fc-46ca-8a1a-9266f878793e None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:20.316494 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778b0c48-e6b4-4b83-8b72-b2db06220053 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:20.330243 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:20.352748 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ac2764-d2cd-4b16-aef3-d9bf845f2c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:20.353366 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ac2764-d2cd-4b16-aef3-d9bf845f2c0e None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:20.353643 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ac2764-d2cd-4b16-aef3-d9bf845f2c0e None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:20.353919 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ac2764-d2cd-4b16-aef3-d9bf845f2c0e None admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:20.354592 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ac2764-d2cd-4b16-aef3-d9bf845f2c0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:20.355529 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ac2764-d2cd-4b16-aef3-d9bf845f2c0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:20.356938 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ac2764-d2cd-4b16-aef3-d9bf845f2c0e None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:20.374501 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 532/1060] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:01:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:01:20.375818 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ac2764-d2cd-4b16-aef3-d9bf845f2c0e None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:20.407827 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a75b2d-75fc-46ca-8a1a-9266f878793e None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:20.448021 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf97c622-a028-44f9-98d8-ce4b75ea9d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:20.449229 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf97c622-a028-44f9-98d8-ce4b75ea9d5e None admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:20.449508 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf97c622-a028-44f9-98d8-ce4b75ea9d5e None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:20.449778 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf97c622-a028-44f9-98d8-ce4b75ea9d5e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:20.450529 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf97c622-a028-44f9-98d8-ce4b75ea9d5e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:20.451620 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:20.452104 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf97c622-a028-44f9-98d8-ce4b75ea9d5e None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:20.482418 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 529/1061] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:01:20 2022] POST /identity/v3/endpoints => generated 327 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 16:01:20.500939 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ac2764-d2cd-4b16-aef3-d9bf845f2c0e None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:20.548953 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce35806-2d9b-4896-98cd-89a562c3f0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:20.549595 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce35806-2d9b-4896-98cd-89a562c3f0c2 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:20.549868 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce35806-2d9b-4896-98cd-89a562c3f0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:20.550137 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce35806-2d9b-4896-98cd-89a562c3f0c2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:20.559141 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce35806-2d9b-4896-98cd-89a562c3f0c2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:20.560628 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce35806-2d9b-4896-98cd-89a562c3f0c2 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:20.574546 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-7ce35806-2d9b-4896-98cd-89a562c3f0c2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:01:20.574546 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:01:20.574546 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:01:20.574546 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:01:20.574546 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:01:20.574546 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:01:20.574546 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:01:20.574546 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:01:20.574546 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:01:20.574546 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:01:20.574546 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:01:20.574546 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:01:20.574546 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 16:01:20.574546 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 16:01:20.574546 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 16:01:20.575676 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 16:01:20.575676 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:01:20.575676 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:01:20.575676 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:01:20.575676 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:01:20.575676 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:01:20.575676 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:01:20.575676 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:01:20.575676 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:01:20.575676 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:01:20.575676 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:01:20.575676 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:01:20.575676 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:01:20.575676 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:01:20.575676 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:01:20.575676 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:01:20.575676 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:01:20.577020 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 16:01:20.577020 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 16:01:20.577020 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 16:01:20.577020 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 16:01:20.577020 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 16:01:20.577020 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 16:01:20.577020 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:01:20.577020 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:01:20.577020 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 533/1062] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 19 16:01:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 16:01:20.582139 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf97c622-a028-44f9-98d8-ce4b75ea9d5e None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:20.700271 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf97c622-a028-44f9-98d8-ce4b75ea9d5e None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:20.702182 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:20.716116 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ea610c-230c-47b5-ac39-df3609fb842a None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:20.716700 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ea610c-230c-47b5-ac39-df3609fb842a None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:20.717201 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ea610c-230c-47b5-ac39-df3609fb842a None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:20.718856 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 534/1063] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:01:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 16:01:20.724965 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a77e5a-f511-4ae4-9135-017886e695bd None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:20.725246 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a77e5a-f511-4ae4-9135-017886e695bd None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:20.725594 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a77e5a-f511-4ae4-9135-017886e695bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:20.741779 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f7d06d-92c4-46ab-b61c-9dec2f70dfcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:20.742431 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7d06d-92c4-46ab-b61c-9dec2f70dfcd None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:20.742714 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7d06d-92c4-46ab-b61c-9dec2f70dfcd None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:20.742982 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7d06d-92c4-46ab-b61c-9dec2f70dfcd None admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:20.743671 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f7d06d-92c4-46ab-b61c-9dec2f70dfcd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:20.743971 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f7d06d-92c4-46ab-b61c-9dec2f70dfcd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:20.746031 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f7d06d-92c4-46ab-b61c-9dec2f70dfcd None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:20.748164 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-f1a77e5a-f511-4ae4-9135-017886e695bd None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:01:20.750225 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-f1a77e5a-f511-4ae4-9135-017886e695bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:01:20.767632 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 530/1064] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 19 16:01:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:20.770190 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f7d06d-92c4-46ab-b61c-9dec2f70dfcd None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:20.829910 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 535/1065] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:01:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 16:01:20.902893 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f7d06d-92c4-46ab-b61c-9dec2f70dfcd None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:20.965320 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a454f0a6-6e6a-4296-a0c5-fdc02d5d6c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:20.966547 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a454f0a6-6e6a-4296-a0c5-fdc02d5d6c07 None admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:20.967060 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a454f0a6-6e6a-4296-a0c5-fdc02d5d6c07 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:20.967572 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a454f0a6-6e6a-4296-a0c5-fdc02d5d6c07 None admin] PATH_INFO: `/v3/projects/be03e5300b3343169f35c643c93d1e06/users/1ecd01cc72834b72b5ff2c4b47249737/roles/c30e151440d44fd18e607056b4d41ec7` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:20.971671 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a454f0a6-6e6a-4296-a0c5-fdc02d5d6c07 None admin] RBAC: Authorizing `identity:create_grant(project_id=be03e5300b3343169f35c643c93d1e06, user_id=1ecd01cc72834b72b5ff2c4b47249737, role_id=c30e151440d44fd18e607056b4d41ec7)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:20.974578 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a454f0a6-6e6a-4296-a0c5-fdc02d5d6c07 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:20.993794 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 531/1066] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Sep 19 16:01:20 2022] PUT /identity/v3/projects/be03e5300b3343169f35c643c93d1e06/users/1ecd01cc72834b72b5ff2c4b47249737/roles/c30e151440d44fd18e607056b4d41ec7 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 16:01:21.111385 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a77e5a-f511-4ae4-9135-017886e695bd None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:21.181409 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:21.235420 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a77e5a-f511-4ae4-9135-017886e695bd None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:21.270433 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76742c2b-477d-4f72-9ec9-d8368d6ef142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:21.271074 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76742c2b-477d-4f72-9ec9-d8368d6ef142 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:21.271353 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76742c2b-477d-4f72-9ec9-d8368d6ef142 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:21.271627 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76742c2b-477d-4f72-9ec9-d8368d6ef142 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:21.282139 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76742c2b-477d-4f72-9ec9-d8368d6ef142 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 16:01:21.287090 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-76742c2b-477d-4f72-9ec9-d8368d6ef142 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 16:01:21.287090 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:01:21.287090 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:01:21.287090 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:01:21.287090 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:01:21.287090 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:01:21.287090 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:01:21.287090 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:01:21.287090 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:01:21.287090 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:01:21.287090 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:01:21.287090 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:01:21.287090 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 16:01:21.287090 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 16:01:21.287090 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 16:01:21.287090 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 16:01:21.288011 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 16:01:21.288011 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 16:01:21.288011 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 16:01:21.288011 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 16:01:21.288011 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:01:21.288011 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:01:21.288011 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:01:21.288011 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:01:21.288011 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:01:21.288011 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:01:21.288011 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:01:21.288011 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:01:21.288011 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:01:21.288011 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:01:21.288011 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:01:21.288011 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:01:21.291114 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:01:21.291114 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:01:21.291114 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:01:21.291114 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:01:21.291114 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 16:01:21.291114 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 16:01:21.291114 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 16:01:21.291114 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 16:01:21.291114 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 16:01:21.291114 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 16:01:21.291114 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 16:01:21.291114 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:01:21.291114 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 536/1067] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 19 16:01:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 16:01:21.295795 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a454f0a6-6e6a-4296-a0c5-fdc02d5d6c07 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:21.323414 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:21.405022 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a454f0a6-6e6a-4296-a0c5-fdc02d5d6c07 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:21.446748 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940de1a5-ca7e-4304-af9a-716c8e32d3a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:21.447367 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940de1a5-ca7e-4304-af9a-716c8e32d3a7 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:21.447643 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940de1a5-ca7e-4304-af9a-716c8e32d3a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:21.447914 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940de1a5-ca7e-4304-af9a-716c8e32d3a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:21.449150 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940de1a5-ca7e-4304-af9a-716c8e32d3a7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:21.449451 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940de1a5-ca7e-4304-af9a-716c8e32d3a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:21.450825 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940de1a5-ca7e-4304-af9a-716c8e32d3a7 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:21.465467 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 532/1068] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 16:01:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:01:21.467138 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76742c2b-477d-4f72-9ec9-d8368d6ef142 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:21.574163 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76742c2b-477d-4f72-9ec9-d8368d6ef142 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:21.587705 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:21.609512 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8dc03c-d52c-4fdb-885f-4dfcd9542896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:21.610137 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8dc03c-d52c-4fdb-885f-4dfcd9542896 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:21.610442 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8dc03c-d52c-4fdb-885f-4dfcd9542896 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:21.610716 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8dc03c-d52c-4fdb-885f-4dfcd9542896 None admin] PATH_INFO: `/v3/users/d2cbec41255b4b38862c5962e6d9023e` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:21.624911 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8dc03c-d52c-4fdb-885f-4dfcd9542896 None admin] RBAC: Authorizing `identity:get_user(user_id=d2cbec41255b4b38862c5962e6d9023e)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:21.626653 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8dc03c-d52c-4fdb-885f-4dfcd9542896 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:21.628254 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 537/1069] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 19 16:01:21 2022] GET /identity/v3/users/d2cbec41255b4b38862c5962e6d9023e => generated 285 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:21.633992 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940de1a5-ca7e-4304-af9a-716c8e32d3a7 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:21.722624 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:21.739469 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940de1a5-ca7e-4304-af9a-716c8e32d3a7 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:21.780098 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d9b632-08e7-408c-be5b-9964dbe9d3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:21.780743 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d9b632-08e7-408c-be5b-9964dbe9d3d5 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:21.781022 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d9b632-08e7-408c-be5b-9964dbe9d3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:21.781406 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d9b632-08e7-408c-be5b-9964dbe9d3d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:21.852678 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d9b632-08e7-408c-be5b-9964dbe9d3d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:21.854016 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d9b632-08e7-408c-be5b-9964dbe9d3d5 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:21.942558 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:21.966583 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 533/1070] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 19 16:01:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 16:01:21.967993 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8dc03c-d52c-4fdb-885f-4dfcd9542896 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:22.103134 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8dc03c-d52c-4fdb-885f-4dfcd9542896 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:22.138938 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db66abd-2f00-46c0-b281-83542c694b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:22.139576 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db66abd-2f00-46c0-b281-83542c694b8e None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:22.139835 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db66abd-2f00-46c0-b281-83542c694b8e None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:22.140099 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db66abd-2f00-46c0-b281-83542c694b8e None admin] PATH_INFO: `/v3/projects/21f4b8d41df2416d80ef5ff75f6624ed` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:22.141263 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db66abd-2f00-46c0-b281-83542c694b8e None admin] RBAC: Authorizing `identity:get_project(project_id=21f4b8d41df2416d80ef5ff75f6624ed)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:22.142718 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db66abd-2f00-46c0-b281-83542c694b8e None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:22.149422 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 538/1071] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 19 16:01:21 2022] GET /identity/v3/projects/21f4b8d41df2416d80ef5ff75f6624ed => generated 311 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:22.150685 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d9b632-08e7-408c-be5b-9964dbe9d3d5 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:22.262870 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d9b632-08e7-408c-be5b-9964dbe9d3d5 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:22.298017 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0a3852-434f-4afa-90fc-c7b3fa4dc252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:22.298732 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0a3852-434f-4afa-90fc-c7b3fa4dc252 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:22.299125 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0a3852-434f-4afa-90fc-c7b3fa4dc252 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:22.299284 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0a3852-434f-4afa-90fc-c7b3fa4dc252 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:22.299949 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0a3852-434f-4afa-90fc-c7b3fa4dc252 None admin] RBAC: Adding query filter params (role.id=63d2d197bfd6497583e94bfb065ce29b, scope.project.id=21f4b8d41df2416d80ef5ff75f6624ed, user.id=d2cbec41255b4b38862c5962e6d9023e) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:22.300244 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0a3852-434f-4afa-90fc-c7b3fa4dc252 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:22.301695 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0a3852-434f-4afa-90fc-c7b3fa4dc252 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:22.314705 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 534/1072] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 19 16:01:22 2022] GET /identity/v3/role_assignments?user.id=d2cbec41255b4b38862c5962e6d9023e&scope.project.id=21f4b8d41df2416d80ef5ff75f6624ed&role.id=63d2d197bfd6497583e94bfb065ce29b => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:22.833643 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2b2ac-34c3-4f25-868a-db0e626b18cf None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:22.833989 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2b2ac-34c3-4f25-868a-db0e626b18cf None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:22.834189 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2b2ac-34c3-4f25-868a-db0e626b18cf None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:22.835698 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 539/1073] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:01:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:01:22.841596 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c458f0-20b9-4797-8fdd-c39291c0e42b None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:22.841977 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c458f0-20b9-4797-8fdd-c39291c0e42b None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:22.842152 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c458f0-20b9-4797-8fdd-c39291c0e42b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:22.860345 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-65c458f0-20b9-4797-8fdd-c39291c0e42b None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:01:22.861526 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-65c458f0-20b9-4797-8fdd-c39291c0e42b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:01:22.907790 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:22.933655 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 535/1074] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:01:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 16:01:23.200276 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb2b2ac-34c3-4f25-868a-db0e626b18cf None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:23.210099 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b3718c-80d7-487c-86c6-300f99da6601 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:23.210435 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b3718c-80d7-487c-86c6-300f99da6601 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:23.210687 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b3718c-80d7-487c-86c6-300f99da6601 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:23.212035 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 536/1075] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:01:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 16:01:23.218247 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e69c8e4-49d9-4e89-a4d8-43a152463299 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:23.218570 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e69c8e4-49d9-4e89-a4d8-43a152463299 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:23.218837 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e69c8e4-49d9-4e89-a4d8-43a152463299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:23.237249 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-7e69c8e4-49d9-4e89-a4d8-43a152463299 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:01:23.238472 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-7e69c8e4-49d9-4e89-a4d8-43a152463299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:01:23.243571 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:23.313039 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb2b2ac-34c3-4f25-868a-db0e626b18cf None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:23.314124 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 537/1076] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:01:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 16:01:23.348532 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bef17a-788f-4b1f-abea-7b5c43f6c88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:23.349198 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bef17a-788f-4b1f-abea-7b5c43f6c88d None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:23.349477 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bef17a-788f-4b1f-abea-7b5c43f6c88d None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:23.349748 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bef17a-788f-4b1f-abea-7b5c43f6c88d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:23.420160 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bef17a-788f-4b1f-abea-7b5c43f6c88d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:23.421663 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bef17a-788f-4b1f-abea-7b5c43f6c88d None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:23.460047 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:23.526920 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 540/1077] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 19 16:01:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 16:01:23.528283 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e69c8e4-49d9-4e89-a4d8-43a152463299 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:23.582075 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bef17a-788f-4b1f-abea-7b5c43f6c88d None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:23.644290 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e69c8e4-49d9-4e89-a4d8-43a152463299 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:23.653957 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:23.682621 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d723860-a2d3-4462-b1e5-0c8081360f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:23.683434 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d723860-a2d3-4462-b1e5-0c8081360f58 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:23.684139 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d723860-a2d3-4462-b1e5-0c8081360f58 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:23.684410 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d723860-a2d3-4462-b1e5-0c8081360f58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:23.690348 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bef17a-788f-4b1f-abea-7b5c43f6c88d None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:23.695313 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d723860-a2d3-4462-b1e5-0c8081360f58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:23.696746 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d723860-a2d3-4462-b1e5-0c8081360f58 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:23.711799 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-7d723860-a2d3-4462-b1e5-0c8081360f58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:01:23.711799 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:01:23.711799 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:01:23.711799 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:01:23.711799 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:01:23.711799 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:01:23.711799 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:01:23.711799 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:01:23.711799 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:01:23.711799 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:01:23.711799 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:01:23.711799 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:01:23.711799 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:01:23.711799 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:01:23.711799 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:01:23.711799 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:01:23.712851 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:01:23.712851 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:01:23.712851 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:01:23.712851 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:01:23.712851 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:01:23.712851 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:01:23.712851 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:01:23.712851 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:01:23.712851 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:01:23.712851 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:01:23.712851 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:01:23.712851 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:01:23.712851 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:01:23.712851 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:01:23.712851 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:01:23.712851 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:01:23.712851 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:01:23.713750 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:01:23.713750 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:01:23.713750 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:01:23.713750 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 538/1078] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:01:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:01:23.718924 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d723860-a2d3-4462-b1e5-0c8081360f58 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:23.727803 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d4b429-c5b7-4cd0-85bc-63fbf6bbb4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:23.728436 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d4b429-c5b7-4cd0-85bc-63fbf6bbb4e5 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:23.728716 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d4b429-c5b7-4cd0-85bc-63fbf6bbb4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:23.728988 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d4b429-c5b7-4cd0-85bc-63fbf6bbb4e5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:23.736822 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:23.741907 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d4b429-c5b7-4cd0-85bc-63fbf6bbb4e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 16:01:23.747658 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-18d4b429-c5b7-4cd0-85bc-63fbf6bbb4e5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 16:01:23.747658 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:01:23.747658 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:01:23.747658 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:01:23.747658 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:01:23.747658 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:01:23.747658 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:01:23.747658 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:01:23.747658 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:01:23.747658 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:01:23.747658 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:01:23.747658 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:01:23.747658 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 16:01:23.747658 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 16:01:23.747658 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 16:01:23.747658 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 16:01:23.748723 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 16:01:23.748723 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 16:01:23.748723 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 16:01:23.748723 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 16:01:23.748723 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:01:23.748723 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:01:23.748723 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:01:23.748723 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:01:23.748723 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:01:23.748723 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:01:23.748723 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:01:23.748723 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:01:23.748723 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:01:23.748723 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:01:23.748723 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:01:23.748723 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:01:23.749786 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:01:23.749786 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:01:23.749786 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:01:23.749786 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:01:23.749786 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 16:01:23.749786 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 16:01:23.749786 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 16:01:23.749786 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 16:01:23.749786 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 16:01:23.749786 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 16:01:23.749786 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 16:01:23.749786 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:01:23.751793 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 541/1079] 10.222.0.57 () {56 vars in 1120 bytes} [Mon Sep 19 16:01:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 16:01:23.757638 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d4b429-c5b7-4cd0-85bc-63fbf6bbb4e5 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:23.838321 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d723860-a2d3-4462-b1e5-0c8081360f58 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:23.870049 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d4b429-c5b7-4cd0-85bc-63fbf6bbb4e5 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:23.880801 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068f6b04-42d9-4aa9-b893-f822d78729c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:23.881417 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068f6b04-42d9-4aa9-b893-f822d78729c3 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:23.881694 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068f6b04-42d9-4aa9-b893-f822d78729c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:23.881966 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068f6b04-42d9-4aa9-b893-f822d78729c3 None admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:23.882648 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068f6b04-42d9-4aa9-b893-f822d78729c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:23.882954 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068f6b04-42d9-4aa9-b893-f822d78729c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:23.884308 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068f6b04-42d9-4aa9-b893-f822d78729c3 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:23.896560 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 539/1080] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:01:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:01:23.920944 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1204b586-52ac-4438-8cd2-b6346a4d3a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:23.921586 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1204b586-52ac-4438-8cd2-b6346a4d3a7f None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:23.921865 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1204b586-52ac-4438-8cd2-b6346a4d3a7f None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:23.922137 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1204b586-52ac-4438-8cd2-b6346a4d3a7f None admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:23.922950 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1204b586-52ac-4438-8cd2-b6346a4d3a7f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:23.923259 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1204b586-52ac-4438-8cd2-b6346a4d3a7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:23.924679 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1204b586-52ac-4438-8cd2-b6346a4d3a7f None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:23.940296 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 542/1081] 10.222.0.57 () {56 vars in 1115 bytes} [Mon Sep 19 16:01:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:01:23.941647 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068f6b04-42d9-4aa9-b893-f822d78729c3 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:23.970312 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:24.068793 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068f6b04-42d9-4aa9-b893-f822d78729c3 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:24.110247 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74163d5c-7b6a-4c61-8e2f-27b373e85a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:24.110960 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74163d5c-7b6a-4c61-8e2f-27b373e85a60 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:24.111243 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74163d5c-7b6a-4c61-8e2f-27b373e85a60 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:24.111520 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74163d5c-7b6a-4c61-8e2f-27b373e85a60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:24.124432 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74163d5c-7b6a-4c61-8e2f-27b373e85a60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:24.125906 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74163d5c-7b6a-4c61-8e2f-27b373e85a60 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:24.141897 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-74163d5c-7b6a-4c61-8e2f-27b373e85a60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:01:24.141897 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:01:24.141897 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:01:24.141897 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:01:24.141897 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:01:24.141897 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:01:24.141897 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:01:24.141897 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:01:24.141897 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:01:24.141897 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:01:24.141897 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:01:24.141897 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:01:24.141897 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:01:24.141897 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:01:24.141897 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:01:24.141897 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:01:24.143146 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:01:24.143146 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:01:24.143146 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:01:24.143146 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:01:24.143146 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:01:24.143146 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:01:24.143146 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:01:24.143146 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:01:24.143146 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:01:24.143146 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:01:24.143146 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:01:24.143146 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:01:24.143146 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:01:24.143146 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:01:24.143146 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:01:24.143146 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:01:24.143146 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:01:24.144172 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:01:24.144172 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:01:24.144172 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:01:24.144172 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 540/1082] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:01:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:01:24.149042 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1204b586-52ac-4438-8cd2-b6346a4d3a7f None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:24.219010 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:24.267540 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1204b586-52ac-4438-8cd2-b6346a4d3a7f None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:24.302915 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7bb8ec-f167-4326-8e67-466c471ad66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:24.303527 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7bb8ec-f167-4326-8e67-466c471ad66c None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:24.303807 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7bb8ec-f167-4326-8e67-466c471ad66c None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:24.304079 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7bb8ec-f167-4326-8e67-466c471ad66c None admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:24.304725 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7bb8ec-f167-4326-8e67-466c471ad66c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:24.305031 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7bb8ec-f167-4326-8e67-466c471ad66c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:24.306560 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7bb8ec-f167-4326-8e67-466c471ad66c None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:24.318784 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 543/1083] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:01:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:24.324560 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74163d5c-7b6a-4c61-8e2f-27b373e85a60 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:24.402076 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:24.433922 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74163d5c-7b6a-4c61-8e2f-27b373e85a60 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:24.468200 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6eea1d-07f7-4831-9eaf-af09a1125348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:24.468857 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6eea1d-07f7-4831-9eaf-af09a1125348 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:24.469102 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6eea1d-07f7-4831-9eaf-af09a1125348 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:24.469373 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6eea1d-07f7-4831-9eaf-af09a1125348 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:24.470615 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0172a7da-2491-4523-9f67-af9c5436f3c2 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:24.470904 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0172a7da-2491-4523-9f67-af9c5436f3c2 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:24.471185 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0172a7da-2491-4523-9f67-af9c5436f3c2 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:24.472541 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 544/1084] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:01:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:01:24.477125 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6eea1d-07f7-4831-9eaf-af09a1125348 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:24.477668 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ef6ec-ee53-4f82-88a5-9c0dfea992d8 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:24.477955 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ef6ec-ee53-4f82-88a5-9c0dfea992d8 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:24.478263 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ef6ec-ee53-4f82-88a5-9c0dfea992d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:24.478577 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6eea1d-07f7-4831-9eaf-af09a1125348 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:24.490242 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-ba6eea1d-07f7-4831-9eaf-af09a1125348 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 19 16:01:24.490242 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:01:24.490242 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:01:24.490242 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:01:24.490242 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:01:24.490242 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:01:24.490242 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:01:24.490242 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:01:24.490242 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:01:24.490242 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:01:24.490242 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:01:24.490242 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:01:24.490242 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 16:01:24.490242 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 16:01:24.490242 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 16:01:24.490242 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 16:01:24.491341 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:01:24.491341 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:01:24.491341 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 16:01:24.491341 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:01:24.491341 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 16:01:24.491341 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:01:24.491341 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:01:24.491341 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:01:24.491341 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:01:24.491341 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:01:24.491341 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:01:24.491341 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:01:24.491341 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:01:24.491341 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:01:24.491341 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:01:24.491341 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:01:24.491341 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:01:24.491341 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:01:24.492286 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:01:24.492286 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:01:24.492286 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 16:01:24.492286 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 16:01:24.492286 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 16:01:24.492286 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 16:01:24.492286 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 16:01:24.492286 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 16:01:24.492286 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 19 16:01:24.492286 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:01:24.492286 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 541/1085] 10.222.0.57 () {56 vars in 1093 bytes} [Mon Sep 19 16:01:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:01:24.497542 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6eea1d-07f7-4831-9eaf-af09a1125348 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:24.499140 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-608ef6ec-ee53-4f82-88a5-9c0dfea992d8 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:01:24.500346 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-608ef6ec-ee53-4f82-88a5-9c0dfea992d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:01:24.567554 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 545/1086] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:01:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 16:01:24.594427 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6eea1d-07f7-4831-9eaf-af09a1125348 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:24.622115 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:24.627284 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2654e349-31ca-4161-a8ee-3ae08c84c12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:24.627877 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2654e349-31ca-4161-a8ee-3ae08c84c12b None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:24.628153 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2654e349-31ca-4161-a8ee-3ae08c84c12b None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:24.628422 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2654e349-31ca-4161-a8ee-3ae08c84c12b None admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:24.629129 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2654e349-31ca-4161-a8ee-3ae08c84c12b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:24.629430 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2654e349-31ca-4161-a8ee-3ae08c84c12b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:24.630862 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2654e349-31ca-4161-a8ee-3ae08c84c12b None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:24.647306 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 542/1087] 10.222.0.57 () {56 vars in 1133 bytes} [Mon Sep 19 16:01:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:01:24.653584 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608ef6ec-ee53-4f82-88a5-9c0dfea992d8 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:24.701512 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:24.750895 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608ef6ec-ee53-4f82-88a5-9c0dfea992d8 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:24.786636 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4f2caa-730a-4c1c-84a2-df719abf7344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:24.787242 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f2caa-730a-4c1c-84a2-df719abf7344 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:24.787519 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f2caa-730a-4c1c-84a2-df719abf7344 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:24.787788 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f2caa-730a-4c1c-84a2-df719abf7344 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:24.835510 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2654e349-31ca-4161-a8ee-3ae08c84c12b None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:24.855703 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4f2caa-730a-4c1c-84a2-df719abf7344 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:24.857164 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4f2caa-730a-4c1c-84a2-df719abf7344 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:24.904606 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:24.937211 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2654e349-31ca-4161-a8ee-3ae08c84c12b None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:24.957823 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 546/1088] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 19 16:01:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 16:01:24.964773 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4f2caa-730a-4c1c-84a2-df719abf7344 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:24.980260 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e07cb6-59b9-491f-a305-d66196638b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:24.980883 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e07cb6-59b9-491f-a305-d66196638b75 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:24.981160 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e07cb6-59b9-491f-a305-d66196638b75 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:24.981431 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e07cb6-59b9-491f-a305-d66196638b75 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:24.991985 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e07cb6-59b9-491f-a305-d66196638b75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 16:01:24.997383 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-f8e07cb6-59b9-491f-a305-d66196638b75 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 16:01:24.997383 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:01:24.997383 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:01:24.997383 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:01:24.997383 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:01:24.997383 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:01:24.997383 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:01:24.997383 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:01:24.997383 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:01:24.997383 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:01:24.997383 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:01:24.997383 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:01:24.997383 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 16:01:24.997383 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 16:01:24.997383 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 16:01:24.997383 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 16:01:24.998556 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 16:01:24.998556 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 16:01:24.998556 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 16:01:24.998556 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 16:01:24.998556 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:01:24.998556 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:01:24.998556 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:01:24.998556 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:01:24.998556 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:01:24.998556 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:01:24.998556 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:01:24.998556 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:01:24.998556 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:01:24.998556 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:01:24.998556 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:01:24.998556 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:01:24.999578 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:01:24.999578 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:01:24.999578 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:01:24.999578 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:01:24.999578 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 16:01:24.999578 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 16:01:24.999578 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 16:01:24.999578 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 16:01:24.999578 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 16:01:24.999578 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 16:01:24.999578 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 16:01:24.999578 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:01:24.999578 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 543/1089] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 19 16:01:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 16:01:25.004500 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e07cb6-59b9-491f-a305-d66196638b75 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:25.055116 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:25.067718 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4f2caa-730a-4c1c-84a2-df719abf7344 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:25.101708 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44afcfb4-8ec7-4996-b48c-b67fe82f94d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:25.102263 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44afcfb4-8ec7-4996-b48c-b67fe82f94d4 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:25.102566 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44afcfb4-8ec7-4996-b48c-b67fe82f94d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:25.102841 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44afcfb4-8ec7-4996-b48c-b67fe82f94d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:25.104883 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e07cb6-59b9-491f-a305-d66196638b75 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:25.113381 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44afcfb4-8ec7-4996-b48c-b67fe82f94d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:25.115217 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44afcfb4-8ec7-4996-b48c-b67fe82f94d4 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:25.130169 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-44afcfb4-8ec7-4996-b48c-b67fe82f94d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:01:25.130169 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:01:25.130169 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:01:25.130169 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:01:25.130169 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:01:25.130169 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:01:25.130169 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:01:25.130169 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:01:25.130169 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:01:25.130169 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:01:25.130169 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:01:25.130169 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:01:25.130169 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:01:25.130169 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:01:25.130169 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:01:25.130169 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:01:25.131270 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:01:25.131270 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:01:25.131270 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:01:25.131270 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:01:25.131270 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:01:25.131270 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:01:25.131270 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:01:25.131270 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:01:25.131270 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:01:25.131270 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:01:25.131270 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:01:25.131270 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:01:25.131270 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:01:25.131270 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:01:25.131270 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:01:25.131270 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:01:25.131270 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:01:25.132191 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:01:25.132191 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:01:25.132191 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:01:25.132191 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 547/1090] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:01:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 16:01:25.137518 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44afcfb4-8ec7-4996-b48c-b67fe82f94d4 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:25.140330 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:25.144011 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c6eaa5-747a-4837-99b2-3b023b0012c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:25.144619 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c6eaa5-747a-4837-99b2-3b023b0012c2 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:25.144895 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c6eaa5-747a-4837-99b2-3b023b0012c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:25.145163 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c6eaa5-747a-4837-99b2-3b023b0012c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:25.145814 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c6eaa5-747a-4837-99b2-3b023b0012c2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:25.146114 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c6eaa5-747a-4837-99b2-3b023b0012c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:25.147486 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c6eaa5-747a-4837-99b2-3b023b0012c2 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:25.156294 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 544/1091] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 16:01:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:01:25.162028 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c6eaa5-747a-4837-99b2-3b023b0012c2 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:25.243111 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44afcfb4-8ec7-4996-b48c-b67fe82f94d4 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:25.266359 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c6eaa5-747a-4837-99b2-3b023b0012c2 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:25.281179 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:25.282202 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f2afb7-10f5-4f81-94ad-0fac3f22c2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:25.283116 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f2afb7-10f5-4f81-94ad-0fac3f22c2fc None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:25.283355 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f2afb7-10f5-4f81-94ad-0fac3f22c2fc None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:25.283607 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f2afb7-10f5-4f81-94ad-0fac3f22c2fc None admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:25.284418 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f2afb7-10f5-4f81-94ad-0fac3f22c2fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:25.284728 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f2afb7-10f5-4f81-94ad-0fac3f22c2fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:25.286177 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f2afb7-10f5-4f81-94ad-0fac3f22c2fc None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:25.300743 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 548/1092] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:01:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:01:25.306312 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f2afb7-10f5-4f81-94ad-0fac3f22c2fc None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:25.307067 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1800b61-572e-40e8-ba35-19ee682f371a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:25.308389 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1800b61-572e-40e8-ba35-19ee682f371a None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:25.308814 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1800b61-572e-40e8-ba35-19ee682f371a None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:25.309227 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1800b61-572e-40e8-ba35-19ee682f371a None admin] PATH_INFO: `/v3/users/d2cbec41255b4b38862c5962e6d9023e` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:25.310557 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1800b61-572e-40e8-ba35-19ee682f371a None admin] RBAC: Authorizing `identity:get_user(user_id=d2cbec41255b4b38862c5962e6d9023e)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:25.312142 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1800b61-572e-40e8-ba35-19ee682f371a None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:25.318957 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 545/1093] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 19 16:01:25 2022] GET /identity/v3/users/d2cbec41255b4b38862c5962e6d9023e => generated 285 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:01:25.320025 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1800b61-572e-40e8-ba35-19ee682f371a None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:25.382781 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:25.429743 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f2afb7-10f5-4f81-94ad-0fac3f22c2fc None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:25.438762 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1800b61-572e-40e8-ba35-19ee682f371a None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:25.466301 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d59d247-ec83-4353-846d-5690d4232bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:25.466975 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d59d247-ec83-4353-846d-5690d4232bbb None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:25.467281 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d59d247-ec83-4353-846d-5690d4232bbb None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:25.467508 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d59d247-ec83-4353-846d-5690d4232bbb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:25.475478 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d59d247-ec83-4353-846d-5690d4232bbb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:25.475637 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61685d4-d528-457b-acb3-5f4153c654fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:25.476110 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61685d4-d528-457b-acb3-5f4153c654fd None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:25.476397 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61685d4-d528-457b-acb3-5f4153c654fd None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:25.476687 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61685d4-d528-457b-acb3-5f4153c654fd None admin] PATH_INFO: `/v3/projects/21f4b8d41df2416d80ef5ff75f6624ed` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:25.476922 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d59d247-ec83-4353-846d-5690d4232bbb None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:25.477801 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61685d4-d528-457b-acb3-5f4153c654fd None admin] RBAC: Authorizing `identity:get_project(project_id=21f4b8d41df2416d80ef5ff75f6624ed)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:25.479240 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61685d4-d528-457b-acb3-5f4153c654fd None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:25.480924 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 546/1094] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 19 16:01:25 2022] GET /identity/v3/projects/21f4b8d41df2416d80ef5ff75f6624ed => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:25.486871 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61685d4-d528-457b-acb3-5f4153c654fd None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:25.488551 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-8d59d247-ec83-4353-846d-5690d4232bbb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:01:25.488551 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:01:25.488551 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:01:25.488551 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:01:25.488551 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:01:25.488551 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:01:25.488551 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:01:25.488551 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:01:25.488551 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:01:25.488551 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:01:25.488551 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:01:25.488551 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:01:25.488551 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 16:01:25.488551 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 16:01:25.488551 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 16:01:25.489391 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 16:01:25.489391 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:01:25.489391 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:01:25.489391 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:01:25.489391 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:01:25.489391 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:01:25.489391 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:01:25.489391 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:01:25.489391 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:01:25.489391 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:01:25.489391 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:01:25.489391 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:01:25.489391 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:01:25.489391 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:01:25.489391 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:01:25.489391 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:01:25.489391 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:01:25.490312 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 16:01:25.490312 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 16:01:25.490312 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 16:01:25.490312 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 16:01:25.490312 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 16:01:25.490312 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 16:01:25.490312 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:01:25.490312 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:01:25.490312 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 549/1095] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 19 16:01:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:01:25.497072 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d59d247-ec83-4353-846d-5690d4232bbb None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:25.554964 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:25.617402 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d59d247-ec83-4353-846d-5690d4232bbb None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:25.620834 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61685d4-d528-457b-acb3-5f4153c654fd None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:25.640807 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:25.652402 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208b7597-9058-460c-9b52-ac95d4c5bf75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:25.653007 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b7597-9058-460c-9b52-ac95d4c5bf75 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:25.653285 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b7597-9058-460c-9b52-ac95d4c5bf75 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:25.653556 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b7597-9058-460c-9b52-ac95d4c5bf75 None admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:25.654244 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208b7597-9058-460c-9b52-ac95d4c5bf75 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:25.654508 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b6fc50-6bd3-4d39-81fe-925496b1821f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:25.654619 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208b7597-9058-460c-9b52-ac95d4c5bf75 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:25.655164 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b6fc50-6bd3-4d39-81fe-925496b1821f None admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:25.655442 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b6fc50-6bd3-4d39-81fe-925496b1821f None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:25.655716 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b6fc50-6bd3-4d39-81fe-925496b1821f None admin] PATH_INFO: `/v3/projects/21f4b8d41df2416d80ef5ff75f6624ed/users/d2cbec41255b4b38862c5962e6d9023e/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:25.655985 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208b7597-9058-460c-9b52-ac95d4c5bf75 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:25.657627 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b6fc50-6bd3-4d39-81fe-925496b1821f None admin] RBAC: Authorizing `identity:create_grant(project_id=21f4b8d41df2416d80ef5ff75f6624ed, user_id=d2cbec41255b4b38862c5962e6d9023e, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:25.659048 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b6fc50-6bd3-4d39-81fe-925496b1821f None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:25.668740 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 550/1096] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 19 16:01:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:25.672391 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 547/1097] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Sep 19 16:01:25 2022] PUT /identity/v3/projects/21f4b8d41df2416d80ef5ff75f6624ed/users/d2cbec41255b4b38862c5962e6d9023e/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:01:25.674850 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208b7597-9058-460c-9b52-ac95d4c5bf75 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:25.783080 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208b7597-9058-460c-9b52-ac95d4c5bf75 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:25.820342 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a2184f-723e-4560-8b15-523eb24abae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:25.820957 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a2184f-723e-4560-8b15-523eb24abae5 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:25.821232 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a2184f-723e-4560-8b15-523eb24abae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:25.821502 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a2184f-723e-4560-8b15-523eb24abae5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:25.822240 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a2184f-723e-4560-8b15-523eb24abae5 None admin] RBAC: Adding query filter params (role.id=c30e151440d44fd18e607056b4d41ec7, scope.project.id=be03e5300b3343169f35c643c93d1e06, user.id=1ecd01cc72834b72b5ff2c4b47249737) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:25.822574 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a2184f-723e-4560-8b15-523eb24abae5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:25.824015 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a2184f-723e-4560-8b15-523eb24abae5 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:25.830499 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 551/1098] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 19 16:01:25 2022] GET /identity/v3/role_assignments?user.id=1ecd01cc72834b72b5ff2c4b47249737&scope.project.id=be03e5300b3343169f35c643c93d1e06&role.id=c30e151440d44fd18e607056b4d41ec7 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:01:27.828523 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902eb3bd-d0cf-4771-9ebb-41106c7cdbc2 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:27.829096 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902eb3bd-d0cf-4771-9ebb-41106c7cdbc2 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:27.829096 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902eb3bd-d0cf-4771-9ebb-41106c7cdbc2 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:27.830398 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 548/1099] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:01:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:01:27.835978 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9df9bc6-d16e-4f2f-b22c-2db06416f99a None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:27.836404 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9df9bc6-d16e-4f2f-b22c-2db06416f99a None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:27.836633 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9df9bc6-d16e-4f2f-b22c-2db06416f99a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:27.854953 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-f9df9bc6-d16e-4f2f-b22c-2db06416f99a None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:01:27.856142 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-f9df9bc6-d16e-4f2f-b22c-2db06416f99a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:01:27.899328 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:27.924551 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 552/1100] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:01:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 16:01:28.226989 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902eb3bd-d0cf-4771-9ebb-41106c7cdbc2 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:28.331643 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902eb3bd-d0cf-4771-9ebb-41106c7cdbc2 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:28.343348 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:28.373106 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda59193-5e12-4715-bb7a-83ac0c45c34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:28.374022 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda59193-5e12-4715-bb7a-83ac0c45c34b None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:28.374470 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda59193-5e12-4715-bb7a-83ac0c45c34b None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:28.374888 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda59193-5e12-4715-bb7a-83ac0c45c34b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:28.386602 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda59193-5e12-4715-bb7a-83ac0c45c34b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 16:01:28.391378 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-dda59193-5e12-4715-bb7a-83ac0c45c34b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 16:01:28.391378 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:01:28.391378 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:01:28.391378 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:01:28.391378 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:01:28.391378 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:01:28.391378 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:01:28.391378 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:01:28.391378 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:01:28.391378 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:01:28.391378 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:01:28.391378 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:01:28.391378 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 16:01:28.391378 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 16:01:28.391378 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 16:01:28.391378 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 16:01:28.392446 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 16:01:28.392446 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 16:01:28.392446 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 16:01:28.392446 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 16:01:28.392446 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:01:28.392446 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:01:28.392446 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:01:28.392446 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:01:28.392446 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:01:28.392446 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:01:28.392446 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:01:28.392446 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:01:28.392446 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:01:28.392446 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:01:28.392446 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:01:28.392446 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:01:28.393609 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:01:28.393609 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:01:28.393609 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:01:28.393609 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:01:28.393609 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 16:01:28.393609 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 16:01:28.393609 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 16:01:28.393609 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 16:01:28.393609 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 16:01:28.393609 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 16:01:28.393609 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 16:01:28.393609 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:01:28.393609 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 549/1101] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 19 16:01:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:01:28.398716 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9df9bc6-d16e-4f2f-b22c-2db06416f99a None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:28.494640 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9df9bc6-d16e-4f2f-b22c-2db06416f99a None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:28.527558 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8ca4ad-fece-4aec-adbc-45701766d25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:28.528151 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8ca4ad-fece-4aec-adbc-45701766d25b None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:28.528425 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8ca4ad-fece-4aec-adbc-45701766d25b None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:28.528691 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8ca4ad-fece-4aec-adbc-45701766d25b None admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:28.529339 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8ca4ad-fece-4aec-adbc-45701766d25b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:28.529637 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8ca4ad-fece-4aec-adbc-45701766d25b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:28.531018 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8ca4ad-fece-4aec-adbc-45701766d25b None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:28.538921 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 553/1102] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 16:01:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:28.545784 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda59193-5e12-4715-bb7a-83ac0c45c34b None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:28.646581 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda59193-5e12-4715-bb7a-83ac0c45c34b None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:28.681182 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd5dff5-a8e9-416a-ab08-e4eccaca87f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:28.681813 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd5dff5-a8e9-416a-ab08-e4eccaca87f1 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:28.682094 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd5dff5-a8e9-416a-ab08-e4eccaca87f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:28.682395 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd5dff5-a8e9-416a-ab08-e4eccaca87f1 None admin] PATH_INFO: `/v3/users/d2cbec41255b4b38862c5962e6d9023e` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:28.683777 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd5dff5-a8e9-416a-ab08-e4eccaca87f1 None admin] RBAC: Authorizing `identity:get_user(user_id=d2cbec41255b4b38862c5962e6d9023e)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:28.686501 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd5dff5-a8e9-416a-ab08-e4eccaca87f1 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:28.688113 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 550/1103] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 19 16:01:28 2022] GET /identity/v3/users/d2cbec41255b4b38862c5962e6d9023e => generated 285 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:28.694463 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8ca4ad-fece-4aec-adbc-45701766d25b None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:28.736086 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:28.796472 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8ca4ad-fece-4aec-adbc-45701766d25b None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:28.834524 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1595e3c-667f-48b6-9e5f-7ab9a5a61fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:28.835149 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1595e3c-667f-48b6-9e5f-7ab9a5a61fe4 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:28.835428 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1595e3c-667f-48b6-9e5f-7ab9a5a61fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:28.835702 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1595e3c-667f-48b6-9e5f-7ab9a5a61fe4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:28.902493 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1595e3c-667f-48b6-9e5f-7ab9a5a61fe4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:28.903942 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1595e3c-667f-48b6-9e5f-7ab9a5a61fe4 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:28.940725 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:29.000062 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 554/1104] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 19 16:01:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 16:01:29.006641 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd5dff5-a8e9-416a-ab08-e4eccaca87f1 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:29.043091 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:29.106446 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd5dff5-a8e9-416a-ab08-e4eccaca87f1 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:29.139538 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324cb0bd-7033-4d95-bbb7-493288726473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:29.140155 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324cb0bd-7033-4d95-bbb7-493288726473 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:29.140434 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324cb0bd-7033-4d95-bbb7-493288726473 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:29.140719 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324cb0bd-7033-4d95-bbb7-493288726473 None admin] PATH_INFO: `/v3/projects/21f4b8d41df2416d80ef5ff75f6624ed` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:29.141813 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324cb0bd-7033-4d95-bbb7-493288726473 None admin] RBAC: Authorizing `identity:get_project(project_id=21f4b8d41df2416d80ef5ff75f6624ed)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:29.143250 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324cb0bd-7033-4d95-bbb7-493288726473 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:29.144860 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 551/1105] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 19 16:01:29 2022] GET /identity/v3/projects/21f4b8d41df2416d80ef5ff75f6624ed => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:29.151011 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1595e3c-667f-48b6-9e5f-7ab9a5a61fe4 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:29.248279 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1595e3c-667f-48b6-9e5f-7ab9a5a61fe4 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:29.280765 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2a409c-e25a-4399-95ec-84255c4d461f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:29.281364 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2a409c-e25a-4399-95ec-84255c4d461f None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:29.281641 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2a409c-e25a-4399-95ec-84255c4d461f None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:29.281914 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2a409c-e25a-4399-95ec-84255c4d461f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:29.282657 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2a409c-e25a-4399-95ec-84255c4d461f None admin] RBAC: Adding query filter params (role.id=63d2d197bfd6497583e94bfb065ce29b, scope.project.id=21f4b8d41df2416d80ef5ff75f6624ed, user.id=d2cbec41255b4b38862c5962e6d9023e) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:29.282982 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2a409c-e25a-4399-95ec-84255c4d461f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:29.284380 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2a409c-e25a-4399-95ec-84255c4d461f None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:29.290013 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 555/1106] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 19 16:01:29 2022] GET /identity/v3/role_assignments?user.id=d2cbec41255b4b38862c5962e6d9023e&scope.project.id=21f4b8d41df2416d80ef5ff75f6624ed&role.id=63d2d197bfd6497583e94bfb065ce29b => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:31.491591 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7270b60-e718-44cb-ba0e-7394a5f99cd6 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:31.492134 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7270b60-e718-44cb-ba0e-7394a5f99cd6 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:31.492134 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7270b60-e718-44cb-ba0e-7394a5f99cd6 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:31.493449 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 552/1107] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:01:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:01:31.499204 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa9601-2594-416e-9f73-f879a4ab6767 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:31.499587 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa9601-2594-416e-9f73-f879a4ab6767 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:31.499750 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa9601-2594-416e-9f73-f879a4ab6767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:31.505929 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:31.519689 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-b9fa9601-2594-416e-9f73-f879a4ab6767 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:01:31.520882 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-b9fa9601-2594-416e-9f73-f879a4ab6767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:01:31.585401 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 556/1108] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:01:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 16:01:31.861378 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7270b60-e718-44cb-ba0e-7394a5f99cd6 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:31.960493 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7270b60-e718-44cb-ba0e-7394a5f99cd6 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:31.962829 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:31.994888 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c9acdb-80a3-4855-8b5c-e3c7e8dc9379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:31.996128 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c9acdb-80a3-4855-8b5c-e3c7e8dc9379 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:31.996405 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c9acdb-80a3-4855-8b5c-e3c7e8dc9379 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:31.996673 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c9acdb-80a3-4855-8b5c-e3c7e8dc9379 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:31.997785 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c9acdb-80a3-4855-8b5c-e3c7e8dc9379 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:31.999228 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c9acdb-80a3-4855-8b5c-e3c7e8dc9379 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:32.000797 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 553/1109] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:01:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:32.006714 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fa9601-2594-416e-9f73-f879a4ab6767 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:32.103216 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fa9601-2594-416e-9f73-f879a4ab6767 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:32.131825 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:32.137092 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f96cd8-3251-48b4-b93f-24f2422f3674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:32.137692 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f96cd8-3251-48b4-b93f-24f2422f3674 None admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:32.137968 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f96cd8-3251-48b4-b93f-24f2422f3674 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:32.138237 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f96cd8-3251-48b4-b93f-24f2422f3674 None admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:32.138909 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f96cd8-3251-48b4-b93f-24f2422f3674 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:32.140317 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f96cd8-3251-48b4-b93f-24f2422f3674 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:32.161232 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 557/1110] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Sep 19 16:01:32 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:01:34.354664 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f87e8f-d51c-4b9d-85ef-1a51395d4bc2 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:34.355217 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f87e8f-d51c-4b9d-85ef-1a51395d4bc2 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:34.355217 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f87e8f-d51c-4b9d-85ef-1a51395d4bc2 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:34.356511 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 554/1111] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:01:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:01:34.361652 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4863a8eb-8b06-4d22-ba77-2b08d7ee99cc None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:34.362078 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4863a8eb-8b06-4d22-ba77-2b08d7ee99cc None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:34.362177 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4863a8eb-8b06-4d22-ba77-2b08d7ee99cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:34.380322 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-4863a8eb-8b06-4d22-ba77-2b08d7ee99cc None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:01:34.381549 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-4863a8eb-8b06-4d22-ba77-2b08d7ee99cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:01:34.450470 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 558/1112] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:01:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 16:01:34.719428 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f87e8f-d51c-4b9d-85ef-1a51395d4bc2 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:34.817803 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f87e8f-d51c-4b9d-85ef-1a51395d4bc2 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:34.851837 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427c70a2-16ad-4f69-b563-a3172283bf0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:34.852459 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427c70a2-16ad-4f69-b563-a3172283bf0d None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:34.852736 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427c70a2-16ad-4f69-b563-a3172283bf0d None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:34.853006 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427c70a2-16ad-4f69-b563-a3172283bf0d None admin] PATH_INFO: `/v3/roles/a1606b79a2c840238de6860bc69b49b7` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:34.854447 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427c70a2-16ad-4f69-b563-a3172283bf0d None admin] RBAC: Authorizing `identity:get_role(role_id=a1606b79a2c840238de6860bc69b49b7)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:34.855794 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427c70a2-16ad-4f69-b563-a3172283bf0d None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:34.857104 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 555/1113] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 19 16:01:34 2022] GET /identity/v3/roles/a1606b79a2c840238de6860bc69b49b7 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:34.863125 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4863a8eb-8b06-4d22-ba77-2b08d7ee99cc None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:34.958265 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4863a8eb-8b06-4d22-ba77-2b08d7ee99cc None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:34.975767 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:34.993172 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c7a3a-8d93-499e-9207-80bb3bad9d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:34.993801 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c7a3a-8d93-499e-9207-80bb3bad9d34 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:34.994081 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c7a3a-8d93-499e-9207-80bb3bad9d34 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:34.994356 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c7a3a-8d93-499e-9207-80bb3bad9d34 None admin] PATH_INFO: `/v3/users/2778e34c38b24c97879fc4a6b14dd9a3` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:35.007684 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5c7a3a-8d93-499e-9207-80bb3bad9d34 None admin] RBAC: Authorizing `identity:get_user(user_id=2778e34c38b24c97879fc4a6b14dd9a3)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:35.009125 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5c7a3a-8d93-499e-9207-80bb3bad9d34 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:35.011329 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 559/1114] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 19 16:01:34 2022] GET /identity/v3/users/2778e34c38b24c97879fc4a6b14dd9a3 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:35.017358 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427c70a2-16ad-4f69-b563-a3172283bf0d None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:35.059401 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:35.115953 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427c70a2-16ad-4f69-b563-a3172283bf0d None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:35.151055 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adffb5e1-4a60-4f64-9d3c-34813c24416a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:35.151717 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adffb5e1-4a60-4f64-9d3c-34813c24416a None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:35.151990 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adffb5e1-4a60-4f64-9d3c-34813c24416a None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:35.152258 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adffb5e1-4a60-4f64-9d3c-34813c24416a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:35.221152 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adffb5e1-4a60-4f64-9d3c-34813c24416a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:35.222611 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adffb5e1-4a60-4f64-9d3c-34813c24416a None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:35.264541 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:35.321075 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 556/1115] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 19 16:01:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 16:01:35.327695 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c7a3a-8d93-499e-9207-80bb3bad9d34 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:35.425094 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c7a3a-8d93-499e-9207-80bb3bad9d34 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:35.458131 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73afe954-5496-4dd0-aec9-1933c1ae059f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:35.458777 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73afe954-5496-4dd0-aec9-1933c1ae059f None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:35.459059 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73afe954-5496-4dd0-aec9-1933c1ae059f None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:35.459336 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73afe954-5496-4dd0-aec9-1933c1ae059f None admin] PATH_INFO: `/v3/projects/21f4b8d41df2416d80ef5ff75f6624ed` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:35.460454 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73afe954-5496-4dd0-aec9-1933c1ae059f None admin] RBAC: Authorizing `identity:get_project(project_id=21f4b8d41df2416d80ef5ff75f6624ed)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:35.461880 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73afe954-5496-4dd0-aec9-1933c1ae059f None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:35.463534 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 560/1116] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 19 16:01:35 2022] GET /identity/v3/projects/21f4b8d41df2416d80ef5ff75f6624ed => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:35.469676 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adffb5e1-4a60-4f64-9d3c-34813c24416a None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:35.568353 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adffb5e1-4a60-4f64-9d3c-34813c24416a None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:35.604217 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a81cf6-d10e-4025-999b-70bd20ffc556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:35.605392 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a81cf6-d10e-4025-999b-70bd20ffc556 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:35.605671 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a81cf6-d10e-4025-999b-70bd20ffc556 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:35.605940 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a81cf6-d10e-4025-999b-70bd20ffc556 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:35.606705 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a81cf6-d10e-4025-999b-70bd20ffc556 None admin] RBAC: Adding query filter params (role.id=a1606b79a2c840238de6860bc69b49b7, scope.project.id=21f4b8d41df2416d80ef5ff75f6624ed, user.id=2778e34c38b24c97879fc4a6b14dd9a3) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:35.607015 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a81cf6-d10e-4025-999b-70bd20ffc556 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:35.608451 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a81cf6-d10e-4025-999b-70bd20ffc556 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:35.614454 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 557/1117] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 19 16:01:35 2022] GET /identity/v3/role_assignments?user.id=2778e34c38b24c97879fc4a6b14dd9a3&scope.project.id=21f4b8d41df2416d80ef5ff75f6624ed&role.id=a1606b79a2c840238de6860bc69b49b7 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:37.735051 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a9688-2d92-48fd-b49c-5e96df88db89 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:37.735582 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a9688-2d92-48fd-b49c-5e96df88db89 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:37.735582 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a9688-2d92-48fd-b49c-5e96df88db89 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:37.736963 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 561/1118] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:01:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:01:37.742320 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926d2cb3-1308-40c4-9d38-f6b4da164de3 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:37.742793 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926d2cb3-1308-40c4-9d38-f6b4da164de3 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:37.744658 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926d2cb3-1308-40c4-9d38-f6b4da164de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:37.750116 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:37.762733 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-926d2cb3-1308-40c4-9d38-f6b4da164de3 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:01:37.763988 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-926d2cb3-1308-40c4-9d38-f6b4da164de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:01:37.830178 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 558/1119] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:01:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 16:01:38.103669 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7a9688-2d92-48fd-b49c-5e96df88db89 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:38.161880 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:38.215949 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7a9688-2d92-48fd-b49c-5e96df88db89 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:38.248605 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431aa449-bfb0-48a3-bbb9-eb90a99192bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:38.249227 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431aa449-bfb0-48a3-bbb9-eb90a99192bf None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:38.249508 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431aa449-bfb0-48a3-bbb9-eb90a99192bf None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:38.249782 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431aa449-bfb0-48a3-bbb9-eb90a99192bf None admin] PATH_INFO: `/v3/roles/a1606b79a2c840238de6860bc69b49b7` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:38.251184 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431aa449-bfb0-48a3-bbb9-eb90a99192bf None admin] RBAC: Authorizing `identity:get_role(role_id=a1606b79a2c840238de6860bc69b49b7)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:38.252547 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431aa449-bfb0-48a3-bbb9-eb90a99192bf None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:38.253858 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 562/1120] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 19 16:01:38 2022] GET /identity/v3/roles/a1606b79a2c840238de6860bc69b49b7 => generated 223 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:38.259695 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926d2cb3-1308-40c4-9d38-f6b4da164de3 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:38.357203 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926d2cb3-1308-40c4-9d38-f6b4da164de3 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:38.389497 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb86985-37b2-4f16-b99b-3423112b1ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:38.390117 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb86985-37b2-4f16-b99b-3423112b1ba7 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:38.390414 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb86985-37b2-4f16-b99b-3423112b1ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:38.390689 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb86985-37b2-4f16-b99b-3423112b1ba7 None admin] PATH_INFO: `/v3/users/2778e34c38b24c97879fc4a6b14dd9a3` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:38.391797 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb86985-37b2-4f16-b99b-3423112b1ba7 None admin] RBAC: Authorizing `identity:get_user(user_id=2778e34c38b24c97879fc4a6b14dd9a3)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:38.393213 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb86985-37b2-4f16-b99b-3423112b1ba7 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:38.394824 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 559/1121] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 19 16:01:38 2022] GET /identity/v3/users/2778e34c38b24c97879fc4a6b14dd9a3 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:38.400580 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431aa449-bfb0-48a3-bbb9-eb90a99192bf None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:38.500879 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431aa449-bfb0-48a3-bbb9-eb90a99192bf None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:38.513855 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:38.534761 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ee7427-3316-484e-af51-0ddd1e88e1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:38.535388 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee7427-3316-484e-af51-0ddd1e88e1e5 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:38.535666 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee7427-3316-484e-af51-0ddd1e88e1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:38.535940 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee7427-3316-484e-af51-0ddd1e88e1e5 None admin] PATH_INFO: `/v3/projects/21f4b8d41df2416d80ef5ff75f6624ed` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:38.537058 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ee7427-3316-484e-af51-0ddd1e88e1e5 None admin] RBAC: Authorizing `identity:get_project(project_id=21f4b8d41df2416d80ef5ff75f6624ed)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:38.538512 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ee7427-3316-484e-af51-0ddd1e88e1e5 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:38.540145 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 563/1122] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 19 16:01:38 2022] GET /identity/v3/projects/21f4b8d41df2416d80ef5ff75f6624ed => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:38.546359 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb86985-37b2-4f16-b99b-3423112b1ba7 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:38.549181 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:38.645037 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb86985-37b2-4f16-b99b-3423112b1ba7 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:38.677756 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0ffd25-ddab-408d-9cad-722202f6aa6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:38.678423 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ffd25-ddab-408d-9cad-722202f6aa6b None admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:38.678706 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ffd25-ddab-408d-9cad-722202f6aa6b None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:38.678982 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ffd25-ddab-408d-9cad-722202f6aa6b None admin] PATH_INFO: `/v3/projects/21f4b8d41df2416d80ef5ff75f6624ed/users/2778e34c38b24c97879fc4a6b14dd9a3/roles/a1606b79a2c840238de6860bc69b49b7` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:38.681158 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0ffd25-ddab-408d-9cad-722202f6aa6b None admin] RBAC: Authorizing `identity:create_grant(project_id=21f4b8d41df2416d80ef5ff75f6624ed, user_id=2778e34c38b24c97879fc4a6b14dd9a3, role_id=a1606b79a2c840238de6860bc69b49b7)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:38.682633 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0ffd25-ddab-408d-9cad-722202f6aa6b None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:38.694932 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 560/1123] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Sep 19 16:01:38 2022] PUT /identity/v3/projects/21f4b8d41df2416d80ef5ff75f6624ed/users/2778e34c38b24c97879fc4a6b14dd9a3/roles/a1606b79a2c840238de6860bc69b49b7 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:01:40.811727 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07974ea7-2eeb-4447-b209-634499931a13 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:40.812231 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07974ea7-2eeb-4447-b209-634499931a13 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:40.812321 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07974ea7-2eeb-4447-b209-634499931a13 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:40.813610 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 564/1124] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:01:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:01:40.818958 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ecbfe-f55b-476f-9e47-4e9923e63d69 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:40.819402 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ecbfe-f55b-476f-9e47-4e9923e63d69 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:40.819493 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ecbfe-f55b-476f-9e47-4e9923e63d69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:40.837908 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-da6ecbfe-f55b-476f-9e47-4e9923e63d69 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:01:40.839732 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-da6ecbfe-f55b-476f-9e47-4e9923e63d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:01:40.895365 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:40.906866 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 561/1125] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:01:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 16:01:41.189072 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07974ea7-2eeb-4447-b209-634499931a13 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:41.288645 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07974ea7-2eeb-4447-b209-634499931a13 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:41.325915 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cb1c4d-f142-4735-946b-1937587f0416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:41.326574 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb1c4d-f142-4735-946b-1937587f0416 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:41.326853 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb1c4d-f142-4735-946b-1937587f0416 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:41.327128 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb1c4d-f142-4735-946b-1937587f0416 None admin] PATH_INFO: `/v3/roles/a1606b79a2c840238de6860bc69b49b7` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:41.328523 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cb1c4d-f142-4735-946b-1937587f0416 None admin] RBAC: Authorizing `identity:get_role(role_id=a1606b79a2c840238de6860bc69b49b7)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:41.329874 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cb1c4d-f142-4735-946b-1937587f0416 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:41.331224 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 565/1126] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 19 16:01:41 2022] GET /identity/v3/roles/a1606b79a2c840238de6860bc69b49b7 => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:41.337039 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6ecbfe-f55b-476f-9e47-4e9923e63d69 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:41.434173 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6ecbfe-f55b-476f-9e47-4e9923e63d69 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:41.466333 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e20a04-7c10-422c-809d-f106b6448e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:41.466977 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e20a04-7c10-422c-809d-f106b6448e40 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:41.467250 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e20a04-7c10-422c-809d-f106b6448e40 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:41.467517 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e20a04-7c10-422c-809d-f106b6448e40 None admin] PATH_INFO: `/v3/users/2778e34c38b24c97879fc4a6b14dd9a3` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:41.468614 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e20a04-7c10-422c-809d-f106b6448e40 None admin] RBAC: Authorizing `identity:get_user(user_id=2778e34c38b24c97879fc4a6b14dd9a3)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:41.470016 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e20a04-7c10-422c-809d-f106b6448e40 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:41.471613 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 562/1127] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 19 16:01:41 2022] GET /identity/v3/users/2778e34c38b24c97879fc4a6b14dd9a3 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:41.477447 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cb1c4d-f142-4735-946b-1937587f0416 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:41.526179 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:41.575476 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cb1c4d-f142-4735-946b-1937587f0416 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:41.607592 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039e4f85-e129-45b5-9010-55a6e7a7cecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:41.608192 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039e4f85-e129-45b5-9010-55a6e7a7cecb None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:41.608469 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039e4f85-e129-45b5-9010-55a6e7a7cecb None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:41.608742 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039e4f85-e129-45b5-9010-55a6e7a7cecb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:41.674071 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039e4f85-e129-45b5-9010-55a6e7a7cecb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:41.675552 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039e4f85-e129-45b5-9010-55a6e7a7cecb None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:41.720859 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:41.771148 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 566/1128] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 19 16:01:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 16:01:41.777941 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e20a04-7c10-422c-809d-f106b6448e40 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:41.838231 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:41.878980 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e20a04-7c10-422c-809d-f106b6448e40 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:41.911437 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82be9b6-9287-47e0-b696-2be7bbd50ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:41.912058 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82be9b6-9287-47e0-b696-2be7bbd50ab6 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:41.912336 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82be9b6-9287-47e0-b696-2be7bbd50ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:41.912608 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82be9b6-9287-47e0-b696-2be7bbd50ab6 None admin] PATH_INFO: `/v3/projects/21f4b8d41df2416d80ef5ff75f6624ed` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:41.913701 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82be9b6-9287-47e0-b696-2be7bbd50ab6 None admin] RBAC: Authorizing `identity:get_project(project_id=21f4b8d41df2416d80ef5ff75f6624ed)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:41.915170 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82be9b6-9287-47e0-b696-2be7bbd50ab6 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:41.917386 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 563/1129] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 19 16:01:41 2022] GET /identity/v3/projects/21f4b8d41df2416d80ef5ff75f6624ed => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:41.923694 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039e4f85-e129-45b5-9010-55a6e7a7cecb None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:42.018895 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039e4f85-e129-45b5-9010-55a6e7a7cecb None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:42.053085 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd72ecbf-64b8-40d4-8171-47f03dedd5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:42.053692 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72ecbf-64b8-40d4-8171-47f03dedd5bf None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:42.053971 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72ecbf-64b8-40d4-8171-47f03dedd5bf None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:42.054241 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72ecbf-64b8-40d4-8171-47f03dedd5bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:42.055001 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd72ecbf-64b8-40d4-8171-47f03dedd5bf None admin] RBAC: Adding query filter params (role.id=a1606b79a2c840238de6860bc69b49b7, scope.project.id=21f4b8d41df2416d80ef5ff75f6624ed, user.id=2778e34c38b24c97879fc4a6b14dd9a3) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:42.055310 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd72ecbf-64b8-40d4-8171-47f03dedd5bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:42.056742 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd72ecbf-64b8-40d4-8171-47f03dedd5bf None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:42.062766 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 567/1130] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 19 16:01:41 2022] GET /identity/v3/role_assignments?user.id=2778e34c38b24c97879fc4a6b14dd9a3&scope.project.id=21f4b8d41df2416d80ef5ff75f6624ed&role.id=a1606b79a2c840238de6860bc69b49b7 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:44.262884 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f67df-cac7-47e9-96aa-a06f913e3e79 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:44.263436 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f67df-cac7-47e9-96aa-a06f913e3e79 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:44.263436 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f67df-cac7-47e9-96aa-a06f913e3e79 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:44.264741 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 564/1131] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:01:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:01:44.270001 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe922a-1dbd-485e-8c46-ab8b7412aa0c None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:44.270414 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe922a-1dbd-485e-8c46-ab8b7412aa0c None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:44.270590 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe922a-1dbd-485e-8c46-ab8b7412aa0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:44.288987 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-9dbe922a-1dbd-485e-8c46-ab8b7412aa0c None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:01:44.290194 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-9dbe922a-1dbd-485e-8c46-ab8b7412aa0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:01:44.297999 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:44.357031 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 568/1132] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:01:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 16:01:44.627676 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30f67df-cac7-47e9-96aa-a06f913e3e79 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:44.732343 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30f67df-cac7-47e9-96aa-a06f913e3e79 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:44.758120 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:44.766779 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599f752e-e857-4e65-b583-0dbf9ca98ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:44.767416 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f752e-e857-4e65-b583-0dbf9ca98ec4 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:44.767696 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f752e-e857-4e65-b583-0dbf9ca98ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:44.767970 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f752e-e857-4e65-b583-0dbf9ca98ec4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:44.769077 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599f752e-e857-4e65-b583-0dbf9ca98ec4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:44.770524 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599f752e-e857-4e65-b583-0dbf9ca98ec4 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:44.772096 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 565/1133] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:01:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:44.778868 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbe922a-1dbd-485e-8c46-ab8b7412aa0c None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:44.882683 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbe922a-1dbd-485e-8c46-ab8b7412aa0c None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:44.918215 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ad3b7a-6fe7-4dc3-b478-27211de49a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:44.919055 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad3b7a-6fe7-4dc3-b478-27211de49a7c None admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:44.919335 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad3b7a-6fe7-4dc3-b478-27211de49a7c None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:44.919608 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad3b7a-6fe7-4dc3-b478-27211de49a7c None admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:44.920271 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ad3b7a-6fe7-4dc3-b478-27211de49a7c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:44.921721 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ad3b7a-6fe7-4dc3-b478-27211de49a7c None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:44.937100 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:44.950258 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 569/1134] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Sep 19 16:01:44 2022] POST /identity/v3/projects => generated 311 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:01:47.107977 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af78dc-a851-4b67-b229-676dfdfba270 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:47.108478 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af78dc-a851-4b67-b229-676dfdfba270 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:47.108478 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af78dc-a851-4b67-b229-676dfdfba270 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:47.109738 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 566/1135] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:01:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:01:47.115064 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe89bc5-95e1-4b72-9d75-3a9f79b98f1e None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:47.115488 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe89bc5-95e1-4b72-9d75-3a9f79b98f1e None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:47.115624 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe89bc5-95e1-4b72-9d75-3a9f79b98f1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:47.134081 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-7fe89bc5-95e1-4b72-9d75-3a9f79b98f1e None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:01:47.135962 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-7fe89bc5-95e1-4b72-9d75-3a9f79b98f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:01:47.219544 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 570/1136] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:01:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 16:01:47.499332 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6af78dc-a851-4b67-b229-676dfdfba270 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:47.599180 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6af78dc-a851-4b67-b229-676dfdfba270 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:47.636362 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f46104-d497-4ddd-bbdc-a1e7ac15c439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:47.637010 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f46104-d497-4ddd-bbdc-a1e7ac15c439 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:47.637285 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f46104-d497-4ddd-bbdc-a1e7ac15c439 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:47.637555 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f46104-d497-4ddd-bbdc-a1e7ac15c439 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:47.638754 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f46104-d497-4ddd-bbdc-a1e7ac15c439 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:47.640193 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f46104-d497-4ddd-bbdc-a1e7ac15c439 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:47.641811 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 567/1137] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:01:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:47.647936 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe89bc5-95e1-4b72-9d75-3a9f79b98f1e None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:47.752436 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe89bc5-95e1-4b72-9d75-3a9f79b98f1e None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:47.777924 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:47.789698 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f850be1c-97f4-491b-88c2-f71fe52f0eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:47.790316 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f850be1c-97f4-491b-88c2-f71fe52f0eed None admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:47.790626 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f850be1c-97f4-491b-88c2-f71fe52f0eed None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:47.790904 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f850be1c-97f4-491b-88c2-f71fe52f0eed None admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:47.792466 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f850be1c-97f4-491b-88c2-f71fe52f0eed None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:47.793910 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f850be1c-97f4-491b-88c2-f71fe52f0eed None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:47.815896 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 571/1138] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Sep 19 16:01:47 2022] POST /identity/v3/users => generated 286 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:01:50.061578 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9902884a-8b13-4ea9-9baf-287a8d5c5f07 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:50.062096 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9902884a-8b13-4ea9-9baf-287a8d5c5f07 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:50.062096 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9902884a-8b13-4ea9-9baf-287a8d5c5f07 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:50.063409 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 568/1139] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:01:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:01:50.068779 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da8b3c-8482-45a6-8b96-191f3cda0007 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:50.069196 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da8b3c-8482-45a6-8b96-191f3cda0007 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:50.069333 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da8b3c-8482-45a6-8b96-191f3cda0007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:50.088083 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-b3da8b3c-8482-45a6-8b96-191f3cda0007 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:01:50.089289 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-b3da8b3c-8482-45a6-8b96-191f3cda0007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:01:50.155241 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 572/1140] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:01:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 16:01:50.423558 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9902884a-8b13-4ea9-9baf-287a8d5c5f07 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:50.486271 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:50.524108 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9902884a-8b13-4ea9-9baf-287a8d5c5f07 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:50.556845 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee7a89e-1047-4ed0-a722-65e0582ad8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:50.558049 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee7a89e-1047-4ed0-a722-65e0582ad8d0 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:50.558321 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee7a89e-1047-4ed0-a722-65e0582ad8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:50.558616 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee7a89e-1047-4ed0-a722-65e0582ad8d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:50.568242 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee7a89e-1047-4ed0-a722-65e0582ad8d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 16:01:50.572872 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-fee7a89e-1047-4ed0-a722-65e0582ad8d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 16:01:50.572872 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:01:50.572872 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:01:50.572872 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:01:50.572872 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:01:50.572872 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:01:50.572872 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:01:50.572872 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:01:50.572872 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:01:50.572872 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:01:50.572872 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:01:50.572872 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:01:50.572872 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 16:01:50.572872 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 16:01:50.572872 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 16:01:50.572872 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 16:01:50.574038 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 16:01:50.574038 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 16:01:50.574038 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 16:01:50.574038 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 16:01:50.574038 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:01:50.574038 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:01:50.574038 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:01:50.574038 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:01:50.574038 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:01:50.574038 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:01:50.574038 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:01:50.574038 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:01:50.574038 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:01:50.574038 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:01:50.574038 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:01:50.574038 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:01:50.574978 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:01:50.574978 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:01:50.574978 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:01:50.574978 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:01:50.574978 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 16:01:50.574978 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 16:01:50.574978 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 16:01:50.574978 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 16:01:50.574978 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 16:01:50.574978 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 16:01:50.574978 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 16:01:50.574978 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:01:50.574978 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 569/1141] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 19 16:01:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:01:50.580107 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3da8b3c-8482-45a6-8b96-191f3cda0007 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:50.675655 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3da8b3c-8482-45a6-8b96-191f3cda0007 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:50.686319 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:50.709783 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13769e73-c7f9-439b-a28f-1d7d5f07dfca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:50.710404 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13769e73-c7f9-439b-a28f-1d7d5f07dfca None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:50.710679 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13769e73-c7f9-439b-a28f-1d7d5f07dfca None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:50.710949 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13769e73-c7f9-439b-a28f-1d7d5f07dfca None admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:50.711605 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13769e73-c7f9-439b-a28f-1d7d5f07dfca None admin] RBAC: Adding query filter params (name=admin) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:50.711905 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13769e73-c7f9-439b-a28f-1d7d5f07dfca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:50.713276 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13769e73-c7f9-439b-a28f-1d7d5f07dfca None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:50.721028 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 573/1142] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 16:01:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:50.727001 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee7a89e-1047-4ed0-a722-65e0582ad8d0 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:50.825855 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee7a89e-1047-4ed0-a722-65e0582ad8d0 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:50.848954 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:50.860962 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb021123-37ac-47db-ba59-90f480691bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:50.861588 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb021123-37ac-47db-ba59-90f480691bff None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:50.861865 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb021123-37ac-47db-ba59-90f480691bff None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:50.862138 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb021123-37ac-47db-ba59-90f480691bff None admin] PATH_INFO: `/v3/users/9fe2fcc2867d4e86a35849fb50078456` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:50.874846 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb021123-37ac-47db-ba59-90f480691bff None admin] RBAC: Authorizing `identity:get_user(user_id=9fe2fcc2867d4e86a35849fb50078456)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:50.876291 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb021123-37ac-47db-ba59-90f480691bff None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:50.877862 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 570/1143] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 19 16:01:50 2022] GET /identity/v3/users/9fe2fcc2867d4e86a35849fb50078456 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:50.883666 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13769e73-c7f9-439b-a28f-1d7d5f07dfca None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:50.981517 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13769e73-c7f9-439b-a28f-1d7d5f07dfca None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:51.015415 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46ffe96-4cee-4eb5-95ac-54b623e961fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:51.016017 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ffe96-4cee-4eb5-95ac-54b623e961fb None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:51.016293 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ffe96-4cee-4eb5-95ac-54b623e961fb None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:51.016565 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ffe96-4cee-4eb5-95ac-54b623e961fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:51.058390 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:51.083935 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46ffe96-4cee-4eb5-95ac-54b623e961fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:51.085369 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46ffe96-4cee-4eb5-95ac-54b623e961fb None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:51.180294 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 574/1144] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 19 16:01:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 16:01:51.186208 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb021123-37ac-47db-ba59-90f480691bff None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:51.284393 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb021123-37ac-47db-ba59-90f480691bff None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:51.317074 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9e4493-df7d-4396-8119-9be2faf0fe8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:51.317710 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e4493-df7d-4396-8119-9be2faf0fe8c None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:51.317991 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e4493-df7d-4396-8119-9be2faf0fe8c None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:51.318268 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e4493-df7d-4396-8119-9be2faf0fe8c None admin] PATH_INFO: `/v3/projects/63ce3251fdda4f1da26be279b08f536c` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:51.319416 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9e4493-df7d-4396-8119-9be2faf0fe8c None admin] RBAC: Authorizing `identity:get_project(project_id=63ce3251fdda4f1da26be279b08f536c)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:51.320831 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9e4493-df7d-4396-8119-9be2faf0fe8c None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:51.322474 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 571/1145] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 19 16:01:51 2022] GET /identity/v3/projects/63ce3251fdda4f1da26be279b08f536c => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:51.329224 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46ffe96-4cee-4eb5-95ac-54b623e961fb None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:51.397142 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:51.430495 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46ffe96-4cee-4eb5-95ac-54b623e961fb None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:51.464343 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baacd3c-d192-466c-90bd-108fb1de0d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:51.464946 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baacd3c-d192-466c-90bd-108fb1de0d37 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:51.465223 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baacd3c-d192-466c-90bd-108fb1de0d37 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:51.465494 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baacd3c-d192-466c-90bd-108fb1de0d37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:51.466214 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baacd3c-d192-466c-90bd-108fb1de0d37 None admin] RBAC: Adding query filter params (role.id=63d2d197bfd6497583e94bfb065ce29b, scope.project.id=63ce3251fdda4f1da26be279b08f536c, user.id=9fe2fcc2867d4e86a35849fb50078456) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:51.466551 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baacd3c-d192-466c-90bd-108fb1de0d37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:51.467981 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baacd3c-d192-466c-90bd-108fb1de0d37 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:51.473285 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 575/1146] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 19 16:01:51 2022] GET /identity/v3/role_assignments?user.id=9fe2fcc2867d4e86a35849fb50078456&scope.project.id=63ce3251fdda4f1da26be279b08f536c&role.id=63d2d197bfd6497583e94bfb065ce29b => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:53.617673 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3896d006-b38e-42a5-968a-5d536de55337 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:53.618230 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3896d006-b38e-42a5-968a-5d536de55337 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:53.618230 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3896d006-b38e-42a5-968a-5d536de55337 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:53.620229 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 572/1147] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:01:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:01:53.625544 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41663ee2-0a1b-4ecd-ac71-2c8f59831168 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:53.625976 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41663ee2-0a1b-4ecd-ac71-2c8f59831168 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:53.626076 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41663ee2-0a1b-4ecd-ac71-2c8f59831168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:53.645213 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-41663ee2-0a1b-4ecd-ac71-2c8f59831168 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:01:53.646458 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-41663ee2-0a1b-4ecd-ac71-2c8f59831168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:01:53.715145 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 576/1148] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:01:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 16:01:53.993880 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3896d006-b38e-42a5-968a-5d536de55337 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:54.049848 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:54.104473 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3896d006-b38e-42a5-968a-5d536de55337 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:54.140111 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf290f44-a186-4ae1-bc40-eec4f776681c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:54.140748 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf290f44-a186-4ae1-bc40-eec4f776681c None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:54.141029 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf290f44-a186-4ae1-bc40-eec4f776681c None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:54.141306 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf290f44-a186-4ae1-bc40-eec4f776681c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:54.153689 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf290f44-a186-4ae1-bc40-eec4f776681c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 16:01:54.158556 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-cf290f44-a186-4ae1-bc40-eec4f776681c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 16:01:54.158556 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:01:54.158556 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:01:54.158556 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:01:54.158556 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:01:54.158556 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:01:54.158556 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:01:54.158556 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:01:54.158556 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:01:54.158556 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:01:54.158556 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:01:54.158556 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:01:54.158556 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 16:01:54.158556 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 16:01:54.158556 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 16:01:54.158556 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 16:01:54.159645 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 16:01:54.159645 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 16:01:54.159645 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 16:01:54.159645 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 16:01:54.159645 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:01:54.159645 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:01:54.159645 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:01:54.159645 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:01:54.159645 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:01:54.159645 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:01:54.159645 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:01:54.159645 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:01:54.159645 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:01:54.159645 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:01:54.159645 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:01:54.159645 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:01:54.160789 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:01:54.160789 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:01:54.160789 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:01:54.160789 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:01:54.160789 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 16:01:54.160789 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 16:01:54.160789 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 16:01:54.160789 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 16:01:54.160789 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 16:01:54.160789 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 16:01:54.160789 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 16:01:54.160789 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:01:54.160789 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 573/1149] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 19 16:01:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:01:54.165838 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41663ee2-0a1b-4ecd-ac71-2c8f59831168 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:54.231137 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:54.265021 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41663ee2-0a1b-4ecd-ac71-2c8f59831168 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:54.297886 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473d901b-15da-46ee-b0cb-ec678629496a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:54.298512 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473d901b-15da-46ee-b0cb-ec678629496a None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:54.298792 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473d901b-15da-46ee-b0cb-ec678629496a None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:54.299066 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473d901b-15da-46ee-b0cb-ec678629496a None admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:54.299727 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473d901b-15da-46ee-b0cb-ec678629496a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:54.300024 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473d901b-15da-46ee-b0cb-ec678629496a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:54.301430 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473d901b-15da-46ee-b0cb-ec678629496a None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:54.309214 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 577/1150] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 16:01:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:54.315066 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf290f44-a186-4ae1-bc40-eec4f776681c None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:54.414136 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf290f44-a186-4ae1-bc40-eec4f776681c None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:54.424863 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:54.449738 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37419f37-3faa-4040-baa7-ef4e45d100fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:54.450362 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37419f37-3faa-4040-baa7-ef4e45d100fc None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:54.450671 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37419f37-3faa-4040-baa7-ef4e45d100fc None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:54.450951 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37419f37-3faa-4040-baa7-ef4e45d100fc None admin] PATH_INFO: `/v3/users/9fe2fcc2867d4e86a35849fb50078456` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:54.452072 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37419f37-3faa-4040-baa7-ef4e45d100fc None admin] RBAC: Authorizing `identity:get_user(user_id=9fe2fcc2867d4e86a35849fb50078456)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:54.453485 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37419f37-3faa-4040-baa7-ef4e45d100fc None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:54.455086 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 574/1151] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 19 16:01:54 2022] GET /identity/v3/users/9fe2fcc2867d4e86a35849fb50078456 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:01:54.460938 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473d901b-15da-46ee-b0cb-ec678629496a None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:54.565027 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473d901b-15da-46ee-b0cb-ec678629496a None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:54.593367 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:54.602321 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb708b99-5ab0-4c00-90fc-e94109b82bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:54.603110 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb708b99-5ab0-4c00-90fc-e94109b82bdc None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:54.603394 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb708b99-5ab0-4c00-90fc-e94109b82bdc None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:54.603661 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb708b99-5ab0-4c00-90fc-e94109b82bdc None admin] PATH_INFO: `/v3/projects/63ce3251fdda4f1da26be279b08f536c` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:54.604869 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb708b99-5ab0-4c00-90fc-e94109b82bdc None admin] RBAC: Authorizing `identity:get_project(project_id=63ce3251fdda4f1da26be279b08f536c)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:54.606325 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb708b99-5ab0-4c00-90fc-e94109b82bdc None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:54.608059 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 578/1152] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 19 16:01:54 2022] GET /identity/v3/projects/63ce3251fdda4f1da26be279b08f536c => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:54.614357 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37419f37-3faa-4040-baa7-ef4e45d100fc None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:54.719869 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37419f37-3faa-4040-baa7-ef4e45d100fc None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:54.756739 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f48a83f-c497-4c93-975f-c97fdca90331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:54.757408 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f48a83f-c497-4c93-975f-c97fdca90331 None admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:54.757689 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f48a83f-c497-4c93-975f-c97fdca90331 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:54.757965 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f48a83f-c497-4c93-975f-c97fdca90331 None admin] PATH_INFO: `/v3/projects/63ce3251fdda4f1da26be279b08f536c/users/9fe2fcc2867d4e86a35849fb50078456/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:54.760147 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f48a83f-c497-4c93-975f-c97fdca90331 None admin] RBAC: Authorizing `identity:create_grant(project_id=63ce3251fdda4f1da26be279b08f536c, user_id=9fe2fcc2867d4e86a35849fb50078456, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:54.762093 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f48a83f-c497-4c93-975f-c97fdca90331 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:54.776598 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 575/1153] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Sep 19 16:01:54 2022] PUT /identity/v3/projects/63ce3251fdda4f1da26be279b08f536c/users/9fe2fcc2867d4e86a35849fb50078456/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:01:56.954270 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bf8073-9815-42f4-a93f-cb548d052b97 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:56.954845 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bf8073-9815-42f4-a93f-cb548d052b97 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:56.954845 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bf8073-9815-42f4-a93f-cb548d052b97 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:56.956181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 579/1154] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:01:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:01:56.961486 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307344cb-c6a2-4a8f-8233-befbd9c0be5c None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:56.961911 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307344cb-c6a2-4a8f-8233-befbd9c0be5c None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:56.962033 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307344cb-c6a2-4a8f-8233-befbd9c0be5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:56.979750 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-307344cb-c6a2-4a8f-8233-befbd9c0be5c None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:01:56.980922 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-307344cb-c6a2-4a8f-8233-befbd9c0be5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:01:57.009581 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:57.048341 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 576/1155] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:01:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 16:01:57.313346 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bf8073-9815-42f4-a93f-cb548d052b97 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:57.412513 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bf8073-9815-42f4-a93f-cb548d052b97 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:57.444979 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd68ad77-da44-4de2-b1c7-18785813171d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:57.445603 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd68ad77-da44-4de2-b1c7-18785813171d None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:57.445878 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd68ad77-da44-4de2-b1c7-18785813171d None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:57.446145 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd68ad77-da44-4de2-b1c7-18785813171d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:57.456548 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd68ad77-da44-4de2-b1c7-18785813171d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 16:01:57.461243 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-fd68ad77-da44-4de2-b1c7-18785813171d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 16:01:57.461243 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:01:57.461243 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:01:57.461243 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:01:57.461243 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:01:57.461243 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:01:57.461243 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:01:57.461243 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:01:57.461243 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:01:57.461243 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:01:57.461243 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:01:57.461243 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:01:57.461243 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 16:01:57.461243 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 16:01:57.461243 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 16:01:57.461243 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 16:01:57.462332 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 16:01:57.462332 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 16:01:57.462332 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 16:01:57.462332 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 16:01:57.462332 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:01:57.462332 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:01:57.462332 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:01:57.462332 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:01:57.462332 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:01:57.462332 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:01:57.462332 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:01:57.462332 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:01:57.462332 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:01:57.462332 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:01:57.462332 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:01:57.462332 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:01:57.463949 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:01:57.463949 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:01:57.463949 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:01:57.463949 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:01:57.463949 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 16:01:57.463949 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 16:01:57.463949 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 16:01:57.463949 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 16:01:57.463949 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 16:01:57.463949 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 16:01:57.463949 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 16:01:57.463949 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:01:57.463949 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 580/1156] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 19 16:01:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:01:57.472521 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307344cb-c6a2-4a8f-8233-befbd9c0be5c None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:57.565790 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307344cb-c6a2-4a8f-8233-befbd9c0be5c None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:57.598945 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68e4e1f-9638-437d-acfd-c456a4f773c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:57.599537 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68e4e1f-9638-437d-acfd-c456a4f773c2 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:57.599810 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68e4e1f-9638-437d-acfd-c456a4f773c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:57.600077 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68e4e1f-9638-437d-acfd-c456a4f773c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:57.600721 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68e4e1f-9638-437d-acfd-c456a4f773c2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:57.601019 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68e4e1f-9638-437d-acfd-c456a4f773c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:57.602351 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68e4e1f-9638-437d-acfd-c456a4f773c2 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:57.610656 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 577/1157] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 16:01:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:57.616617 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd68ad77-da44-4de2-b1c7-18785813171d None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:57.670987 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:57.714954 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd68ad77-da44-4de2-b1c7-18785813171d None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:57.748826 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d069bf-d1aa-484f-bc93-383d4ce61e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:57.749453 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d069bf-d1aa-484f-bc93-383d4ce61e62 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:57.749734 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d069bf-d1aa-484f-bc93-383d4ce61e62 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:57.750008 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d069bf-d1aa-484f-bc93-383d4ce61e62 None admin] PATH_INFO: `/v3/users/9fe2fcc2867d4e86a35849fb50078456` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:57.751281 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d069bf-d1aa-484f-bc93-383d4ce61e62 None admin] RBAC: Authorizing `identity:get_user(user_id=9fe2fcc2867d4e86a35849fb50078456)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:57.752721 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d069bf-d1aa-484f-bc93-383d4ce61e62 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:57.754310 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 581/1158] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 19 16:01:57 2022] GET /identity/v3/users/9fe2fcc2867d4e86a35849fb50078456 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:57.760202 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68e4e1f-9638-437d-acfd-c456a4f773c2 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:57.792303 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:57.864267 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68e4e1f-9638-437d-acfd-c456a4f773c2 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:57.896922 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b251c08-999f-495e-a819-443ade31d2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:57.897519 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b251c08-999f-495e-a819-443ade31d2f1 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:57.897795 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b251c08-999f-495e-a819-443ade31d2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:57.898066 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b251c08-999f-495e-a819-443ade31d2f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:57.964243 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b251c08-999f-495e-a819-443ade31d2f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:57.965686 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b251c08-999f-495e-a819-443ade31d2f1 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:57.995681 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:58.067382 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 578/1159] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 19 16:01:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 16:01:58.074087 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d069bf-d1aa-484f-bc93-383d4ce61e62 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:58.177024 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d069bf-d1aa-484f-bc93-383d4ce61e62 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:58.200056 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:58.211033 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1160704c-7cc7-44df-b80e-b9d82419e5c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:58.211655 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1160704c-7cc7-44df-b80e-b9d82419e5c0 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:58.211936 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1160704c-7cc7-44df-b80e-b9d82419e5c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:58.212211 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1160704c-7cc7-44df-b80e-b9d82419e5c0 None admin] PATH_INFO: `/v3/projects/63ce3251fdda4f1da26be279b08f536c` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:58.213967 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1160704c-7cc7-44df-b80e-b9d82419e5c0 None admin] RBAC: Authorizing `identity:get_project(project_id=63ce3251fdda4f1da26be279b08f536c)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:58.215428 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1160704c-7cc7-44df-b80e-b9d82419e5c0 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:58.217064 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 582/1160] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 19 16:01:58 2022] GET /identity/v3/projects/63ce3251fdda4f1da26be279b08f536c => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:01:58.223251 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b251c08-999f-495e-a819-443ade31d2f1 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:01:58.328668 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b251c08-999f-495e-a819-443ade31d2f1 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:01:58.358439 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:01:58.364671 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da7b3db-d144-447a-a642-abad9f199b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:01:58.365290 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da7b3db-d144-447a-a642-abad9f199b5d None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:01:58.365569 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da7b3db-d144-447a-a642-abad9f199b5d None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:01:58.365843 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da7b3db-d144-447a-a642-abad9f199b5d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:01:58.366609 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da7b3db-d144-447a-a642-abad9f199b5d None admin] RBAC: Adding query filter params (role.id=63d2d197bfd6497583e94bfb065ce29b, scope.project.id=63ce3251fdda4f1da26be279b08f536c, user.id=9fe2fcc2867d4e86a35849fb50078456) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:01:58.366917 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da7b3db-d144-447a-a642-abad9f199b5d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:01:58.368345 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da7b3db-d144-447a-a642-abad9f199b5d None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:01:58.375347 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 579/1161] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 19 16:01:58 2022] GET /identity/v3/role_assignments?user.id=9fe2fcc2867d4e86a35849fb50078456&scope.project.id=63ce3251fdda4f1da26be279b08f536c&role.id=63d2d197bfd6497583e94bfb065ce29b => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:02:00.654686 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a608bfb4-4041-4f9e-802a-a8d8af93c29c None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:02:00.655216 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a608bfb4-4041-4f9e-802a-a8d8af93c29c None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:02:00.655216 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a608bfb4-4041-4f9e-802a-a8d8af93c29c None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:02:00.656568 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 583/1162] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:02:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:02:00.661798 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa56f3-fe5e-420e-b542-c99706e53e27 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:02:00.662216 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa56f3-fe5e-420e-b542-c99706e53e27 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:02:00.662343 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa56f3-fe5e-420e-b542-c99706e53e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:02:00.680952 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-65fa56f3-fe5e-420e-b542-c99706e53e27 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:02:00.682670 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-65fa56f3-fe5e-420e-b542-c99706e53e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:02:00.746191 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 580/1163] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:02:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 16:02:01.020897 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a608bfb4-4041-4f9e-802a-a8d8af93c29c None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:02:01.126942 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a608bfb4-4041-4f9e-802a-a8d8af93c29c None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:02:01.160834 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055cf485-4c36-4af7-9bfa-ba9f8cdb9b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:02:01.161444 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055cf485-4c36-4af7-9bfa-ba9f8cdb9b43 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:02:01.161726 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055cf485-4c36-4af7-9bfa-ba9f8cdb9b43 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:02:01.162003 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055cf485-4c36-4af7-9bfa-ba9f8cdb9b43 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:02:01.221817 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:02:01.232091 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055cf485-4c36-4af7-9bfa-ba9f8cdb9b43 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:02:01.233549 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055cf485-4c36-4af7-9bfa-ba9f8cdb9b43 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:02:01.334706 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 584/1164] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 19 16:02:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 16:02:01.341577 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fa56f3-fe5e-420e-b542-c99706e53e27 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:02:01.438435 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fa56f3-fe5e-420e-b542-c99706e53e27 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:02:01.460965 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:02:01.472638 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a0fed4-1ccf-45b7-9f46-6044fe1d99e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:02:01.473259 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a0fed4-1ccf-45b7-9f46-6044fe1d99e6 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:02:01.473537 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a0fed4-1ccf-45b7-9f46-6044fe1d99e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:02:01.473804 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a0fed4-1ccf-45b7-9f46-6044fe1d99e6 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:02:01.480955 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a0fed4-1ccf-45b7-9f46-6044fe1d99e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:02:01.482363 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a0fed4-1ccf-45b7-9f46-6044fe1d99e6 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:02:01.493430 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-49a0fed4-1ccf-45b7-9f46-6044fe1d99e6 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 19 16:02:01.493430 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:02:01.493430 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 19 16:02:01.493430 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 19 16:02:01.493430 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 16:02:01.493430 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 16:02:01.493430 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 16:02:01.493430 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 16:02:01.493430 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 19 16:02:01.493430 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:02:01.493430 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 16:02:01.493430 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:02:01.493430 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:02:01.493430 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:02:01.493430 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:02:01.493430 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:02:01.493430 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:02:01.494652 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:02:01.494652 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:02:01.494652 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:02:01.494652 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:02:01.494652 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:02:01.494652 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:02:01.494652 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:02:01.494652 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:02:01.494652 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:02:01.494652 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:02:01.494652 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:02:01.494652 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:02:01.494652 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:02:01.494652 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:02:01.494652 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:02:01.494652 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:02:01.495499 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:02:01.495499 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:02:01.495499 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:02:01.495499 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:02:01.495499 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:02:01.495499 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:02:01.495499 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:02:01.495499 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:02:01.495499 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:02:01.495499 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:02:01.495499 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 19 16:02:01.495499 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:02:01.495499 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 19 16:02:01.495499 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:02:01.495499 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 581/1165] 10.222.0.57 () {56 vars in 1117 bytes} [Mon Sep 19 16:02:01 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:02:01.504120 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055cf485-4c36-4af7-9bfa-ba9f8cdb9b43 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:02:01.592707 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:02:01.601824 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055cf485-4c36-4af7-9bfa-ba9f8cdb9b43 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:02:01.634972 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4a980f-2a14-4da4-ab9c-6c25bf4441bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:02:01.635911 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a980f-2a14-4da4-ab9c-6c25bf4441bb None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:02:01.636400 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a980f-2a14-4da4-ab9c-6c25bf4441bb None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:02:01.636888 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a980f-2a14-4da4-ab9c-6c25bf4441bb None admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:02:01.637957 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4a980f-2a14-4da4-ab9c-6c25bf4441bb None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:02:01.638348 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4a980f-2a14-4da4-ab9c-6c25bf4441bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:02:01.640072 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4a980f-2a14-4da4-ab9c-6c25bf4441bb None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:02:01.648025 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 585/1166] 10.222.0.57 () {56 vars in 1115 bytes} [Mon Sep 19 16:02:01 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:02:03.794907 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1959645-ff53-464a-bfb0-47c17cecd507 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:02:03.795407 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1959645-ff53-464a-bfb0-47c17cecd507 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:02:03.795502 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1959645-ff53-464a-bfb0-47c17cecd507 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:02:03.796770 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 582/1167] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:02:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:02:03.801949 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ab8c7-4829-49fa-80d6-3ed399e1e09e None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:02:03.802407 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ab8c7-4829-49fa-80d6-3ed399e1e09e None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:02:03.802602 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ab8c7-4829-49fa-80d6-3ed399e1e09e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:02:03.820962 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-3b0ab8c7-4829-49fa-80d6-3ed399e1e09e None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:02:03.822188 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-3b0ab8c7-4829-49fa-80d6-3ed399e1e09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:02:03.888002 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 586/1168] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:02:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 16:02:04.168048 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1959645-ff53-464a-bfb0-47c17cecd507 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:02:04.270691 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1959645-ff53-464a-bfb0-47c17cecd507 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:02:04.303911 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817ca5b6-2bd9-4aa1-86de-8e64145ae379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:02:04.304514 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ca5b6-2bd9-4aa1-86de-8e64145ae379 None admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:02:04.304793 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ca5b6-2bd9-4aa1-86de-8e64145ae379 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:02:04.305090 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ca5b6-2bd9-4aa1-86de-8e64145ae379 None admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:02:04.305753 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817ca5b6-2bd9-4aa1-86de-8e64145ae379 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:02:04.307680 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817ca5b6-2bd9-4aa1-86de-8e64145ae379 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:02:04.323739 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 583/1169] 10.222.0.57 () {60 vars in 1138 bytes} [Mon Sep 19 16:02:04 2022] POST /identity/v3/domains => generated 267 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:02:06.694013 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ecc149-95cb-4811-9c84-0b3d0ed5ab71 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:02:06.694577 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ecc149-95cb-4811-9c84-0b3d0ed5ab71 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:02:06.694670 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ecc149-95cb-4811-9c84-0b3d0ed5ab71 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:02:06.696295 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 587/1170] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:02:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:02:06.702578 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff836a4-e5af-4372-b7d0-76bc49092965 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:02:06.703001 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff836a4-e5af-4372-b7d0-76bc49092965 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:02:06.703228 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff836a4-e5af-4372-b7d0-76bc49092965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:02:06.722998 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-8ff836a4-e5af-4372-b7d0-76bc49092965 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:02:06.724443 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-8ff836a4-e5af-4372-b7d0-76bc49092965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:02:06.769277 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:02:06.800280 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 584/1171] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:02:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 16:02:07.065898 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ecc149-95cb-4811-9c84-0b3d0ed5ab71 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:02:07.157509 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:02:07.176579 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ecc149-95cb-4811-9c84-0b3d0ed5ab71 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:02:07.215477 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e0677-dc3a-4ced-98c8-7337936f351c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:02:07.216395 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e0677-dc3a-4ced-98c8-7337936f351c None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:02:07.216900 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e0677-dc3a-4ced-98c8-7337936f351c None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:02:07.217228 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e0677-dc3a-4ced-98c8-7337936f351c None admin] PATH_INFO: `/v3/domains/424d389444354b76973500da02af1f5d` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:02:07.230059 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607e0677-dc3a-4ced-98c8-7337936f351c None admin] RBAC: Authorizing `identity:get_domain(domain_id=424d389444354b76973500da02af1f5d)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:02:07.231539 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607e0677-dc3a-4ced-98c8-7337936f351c None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:02:07.233197 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 588/1172] 10.222.0.57 () {56 vars in 1183 bytes} [Mon Sep 19 16:02:07 2022] GET /identity/v3/domains/424d389444354b76973500da02af1f5d => generated 267 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:02:07.239333 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff836a4-e5af-4372-b7d0-76bc49092965 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:02:07.339854 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff836a4-e5af-4372-b7d0-76bc49092965 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:02:07.374533 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be494c5-fdf3-467b-976f-41316de2cdd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:02:07.375132 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be494c5-fdf3-467b-976f-41316de2cdd7 None admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:02:07.375409 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be494c5-fdf3-467b-976f-41316de2cdd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:02:07.375677 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be494c5-fdf3-467b-976f-41316de2cdd7 None admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:02:07.376313 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be494c5-fdf3-467b-976f-41316de2cdd7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:02:07.377725 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be494c5-fdf3-467b-976f-41316de2cdd7 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:02:07.404488 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 585/1173] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Sep 19 16:02:07 2022] POST /identity/v3/projects => generated 361 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:02:09.919408 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3b5425-e37b-4b41-a11b-8943d675d8dd None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:02:09.919982 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3b5425-e37b-4b41-a11b-8943d675d8dd None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:02:09.920087 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3b5425-e37b-4b41-a11b-8943d675d8dd None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:02:09.921970 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 589/1174] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:02:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:02:09.929555 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dfa2ff-829b-4c4c-8069-3d6af4a858aa None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:02:09.930042 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dfa2ff-829b-4c4c-8069-3d6af4a858aa None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:02:09.930335 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dfa2ff-829b-4c4c-8069-3d6af4a858aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:02:09.953405 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-67dfa2ff-829b-4c4c-8069-3d6af4a858aa None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:02:09.954616 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-67dfa2ff-829b-4c4c-8069-3d6af4a858aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:02:09.974160 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:02:10.029555 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 586/1175] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:02:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 16:02:10.319065 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3b5425-e37b-4b41-a11b-8943d675d8dd None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:02:10.424793 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3b5425-e37b-4b41-a11b-8943d675d8dd None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:02:10.458996 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8aeeeb-6d14-4fe6-a56d-2dceaf24858f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:02:10.459612 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8aeeeb-6d14-4fe6-a56d-2dceaf24858f None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:02:10.459890 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8aeeeb-6d14-4fe6-a56d-2dceaf24858f None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:02:10.460154 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8aeeeb-6d14-4fe6-a56d-2dceaf24858f None admin] PATH_INFO: `/v3/domains/424d389444354b76973500da02af1f5d` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:02:10.461272 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8aeeeb-6d14-4fe6-a56d-2dceaf24858f None admin] RBAC: Authorizing `identity:get_domain(domain_id=424d389444354b76973500da02af1f5d)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:02:10.462759 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8aeeeb-6d14-4fe6-a56d-2dceaf24858f None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:02:10.464825 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 590/1176] 10.222.0.57 () {56 vars in 1183 bytes} [Mon Sep 19 16:02:10 2022] GET /identity/v3/domains/424d389444354b76973500da02af1f5d => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:02:10.470780 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dfa2ff-829b-4c4c-8069-3d6af4a858aa None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:02:10.569781 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dfa2ff-829b-4c4c-8069-3d6af4a858aa None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:02:10.607299 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7acabd7-de52-4bbd-b22b-f30d3024c441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:02:10.608027 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7acabd7-de52-4bbd-b22b-f30d3024c441 None admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:02:10.608364 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7acabd7-de52-4bbd-b22b-f30d3024c441 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:02:10.608701 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7acabd7-de52-4bbd-b22b-f30d3024c441 None admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:02:10.609422 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7acabd7-de52-4bbd-b22b-f30d3024c441 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:02:10.610929 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7acabd7-de52-4bbd-b22b-f30d3024c441 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:02:10.633375 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 587/1177] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Sep 19 16:02:10 2022] POST /identity/v3/users => generated 311 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 16:02:12.966999 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b66c218-0c6e-48a8-bd7d-8e1fa4a08d0f None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:02:12.967549 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b66c218-0c6e-48a8-bd7d-8e1fa4a08d0f None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:02:12.967666 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b66c218-0c6e-48a8-bd7d-8e1fa4a08d0f None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:02:12.968990 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 591/1178] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:02:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:02:12.974248 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de87bc4a-4af3-42c7-b1ec-65cbd79a1a18 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:02:12.974743 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de87bc4a-4af3-42c7-b1ec-65cbd79a1a18 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:02:12.974853 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de87bc4a-4af3-42c7-b1ec-65cbd79a1a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:02:12.992885 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-de87bc4a-4af3-42c7-b1ec-65cbd79a1a18 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:02:12.994605 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-de87bc4a-4af3-42c7-b1ec-65cbd79a1a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:02:13.002248 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:02:13.064451 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 588/1179] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:02:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 16:02:13.332319 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b66c218-0c6e-48a8-bd7d-8e1fa4a08d0f None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:02:13.348368 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:02:13.442064 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b66c218-0c6e-48a8-bd7d-8e1fa4a08d0f None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:02:13.479957 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a96ccd3-735b-4b64-ba12-76c5a34b7194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:02:13.480587 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96ccd3-735b-4b64-ba12-76c5a34b7194 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:02:13.480866 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96ccd3-735b-4b64-ba12-76c5a34b7194 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:02:13.481140 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96ccd3-735b-4b64-ba12-76c5a34b7194 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:02:13.492007 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a96ccd3-735b-4b64-ba12-76c5a34b7194 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 16:02:13.497560 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-6a96ccd3-735b-4b64-ba12-76c5a34b7194 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 16:02:13.497560 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:02:13.497560 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:02:13.497560 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:02:13.497560 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:02:13.497560 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:02:13.497560 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:02:13.497560 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:02:13.497560 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:02:13.497560 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:02:13.497560 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:02:13.497560 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:02:13.497560 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 16:02:13.497560 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 16:02:13.497560 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 16:02:13.497560 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 16:02:13.498762 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 16:02:13.498762 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 16:02:13.498762 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 16:02:13.498762 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 16:02:13.498762 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:02:13.498762 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:02:13.498762 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:02:13.498762 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:02:13.498762 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:02:13.498762 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:02:13.498762 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:02:13.498762 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:02:13.498762 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:02:13.498762 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:02:13.498762 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:02:13.498762 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:02:13.500087 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:02:13.500087 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:02:13.500087 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:02:13.500087 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:02:13.500087 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 16:02:13.500087 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 16:02:13.500087 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 16:02:13.500087 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 16:02:13.500087 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 16:02:13.500087 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 16:02:13.500087 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 16:02:13.500087 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:02:13.500087 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 592/1180] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 19 16:02:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:02:13.506102 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de87bc4a-4af3-42c7-b1ec-65cbd79a1a18 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:02:13.621194 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de87bc4a-4af3-42c7-b1ec-65cbd79a1a18 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:02:13.658299 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7f5a15-73fc-422f-85a9-2c7214370f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:02:13.658961 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f5a15-73fc-422f-85a9-2c7214370f2f None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:02:13.659235 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f5a15-73fc-422f-85a9-2c7214370f2f None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:02:13.659505 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f5a15-73fc-422f-85a9-2c7214370f2f None admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:02:13.660160 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7f5a15-73fc-422f-85a9-2c7214370f2f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:02:13.660461 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7f5a15-73fc-422f-85a9-2c7214370f2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:02:13.661794 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7f5a15-73fc-422f-85a9-2c7214370f2f None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:02:13.673004 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 589/1181] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 16:02:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:02:13.679104 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a96ccd3-735b-4b64-ba12-76c5a34b7194 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:02:13.762515 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:02:13.793505 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a96ccd3-735b-4b64-ba12-76c5a34b7194 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:02:13.830183 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f115ec0-ed27-47bb-9033-64fe378cfc21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:02:13.830920 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f115ec0-ed27-47bb-9033-64fe378cfc21 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:02:13.831232 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f115ec0-ed27-47bb-9033-64fe378cfc21 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:02:13.831556 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f115ec0-ed27-47bb-9033-64fe378cfc21 None admin] PATH_INFO: `/v3/users/7eb3fa89ce11499d9ba9e4bb1afeb26f` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:02:13.847329 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f115ec0-ed27-47bb-9033-64fe378cfc21 None admin] RBAC: Authorizing `identity:get_user(user_id=7eb3fa89ce11499d9ba9e4bb1afeb26f)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:02:13.849082 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f115ec0-ed27-47bb-9033-64fe378cfc21 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:02:13.851062 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 593/1182] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 19 16:02:13 2022] GET /identity/v3/users/7eb3fa89ce11499d9ba9e4bb1afeb26f => generated 311 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:02:13.857572 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7f5a15-73fc-422f-85a9-2c7214370f2f None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:02:13.861371 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:02:13.967722 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7f5a15-73fc-422f-85a9-2c7214370f2f None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:02:14.002885 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff567e2-08e0-4bf4-a87e-3b9d5169e83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:02:14.003601 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff567e2-08e0-4bf4-a87e-3b9d5169e83c None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:02:14.003751 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff567e2-08e0-4bf4-a87e-3b9d5169e83c None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:02:14.004022 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff567e2-08e0-4bf4-a87e-3b9d5169e83c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:02:14.076389 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff567e2-08e0-4bf4-a87e-3b9d5169e83c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:02:14.077888 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff567e2-08e0-4bf4-a87e-3b9d5169e83c None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:02:14.080530 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:02:14.176942 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 590/1183] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 19 16:02:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 16:02:14.183063 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f115ec0-ed27-47bb-9033-64fe378cfc21 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:02:14.299633 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f115ec0-ed27-47bb-9033-64fe378cfc21 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:02:14.336391 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:02:14.345661 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abc720e-f157-4549-9b47-431c053904b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:02:14.346298 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc720e-f157-4549-9b47-431c053904b6 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:02:14.346611 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc720e-f157-4549-9b47-431c053904b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:02:14.346887 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc720e-f157-4549-9b47-431c053904b6 None admin] PATH_INFO: `/v3/projects/1cd9310705b540dbb786b4f99fde23d7` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:02:14.348029 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abc720e-f157-4549-9b47-431c053904b6 None admin] RBAC: Authorizing `identity:get_project(project_id=1cd9310705b540dbb786b4f99fde23d7)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:02:14.349512 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abc720e-f157-4549-9b47-431c053904b6 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:02:14.351540 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 594/1184] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 19 16:02:14 2022] GET /identity/v3/projects/1cd9310705b540dbb786b4f99fde23d7 => generated 361 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:02:14.358197 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff567e2-08e0-4bf4-a87e-3b9d5169e83c None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:02:14.463569 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff567e2-08e0-4bf4-a87e-3b9d5169e83c None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:02:14.465935 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:02:14.499498 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7876fb7e-ccd3-4bbb-a28c-88035cd45526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:02:14.500092 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7876fb7e-ccd3-4bbb-a28c-88035cd45526 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:02:14.500395 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7876fb7e-ccd3-4bbb-a28c-88035cd45526 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:02:14.500636 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7876fb7e-ccd3-4bbb-a28c-88035cd45526 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:02:14.501346 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7876fb7e-ccd3-4bbb-a28c-88035cd45526 None admin] RBAC: Adding query filter params (role.id=63d2d197bfd6497583e94bfb065ce29b, scope.project.id=1cd9310705b540dbb786b4f99fde23d7, user.id=7eb3fa89ce11499d9ba9e4bb1afeb26f) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:02:14.501647 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7876fb7e-ccd3-4bbb-a28c-88035cd45526 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:02:14.503077 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7876fb7e-ccd3-4bbb-a28c-88035cd45526 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:02:14.508392 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 591/1185] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 19 16:02:14 2022] GET /identity/v3/role_assignments?user.id=7eb3fa89ce11499d9ba9e4bb1afeb26f&scope.project.id=1cd9310705b540dbb786b4f99fde23d7&role.id=63d2d197bfd6497583e94bfb065ce29b => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:02:16.665755 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b01b665-a7a9-4cc1-aaf8-af3f9a9e71b9 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:02:16.666310 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b01b665-a7a9-4cc1-aaf8-af3f9a9e71b9 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:02:16.666310 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b01b665-a7a9-4cc1-aaf8-af3f9a9e71b9 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:02:16.667923 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 595/1186] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:02:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:02:16.673043 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfeeeee-746b-482c-8765-c0c173d2bfab None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:02:16.673471 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfeeeee-746b-482c-8765-c0c173d2bfab None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:02:16.673596 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfeeeee-746b-482c-8765-c0c173d2bfab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:02:16.693554 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-fcfeeeee-746b-482c-8765-c0c173d2bfab None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:02:16.694871 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-fcfeeeee-746b-482c-8765-c0c173d2bfab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:02:16.765622 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 592/1187] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:02:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 16:02:17.052707 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b01b665-a7a9-4cc1-aaf8-af3f9a9e71b9 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:02:17.157646 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b01b665-a7a9-4cc1-aaf8-af3f9a9e71b9 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:02:17.193032 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da768a60-c8b8-450c-ae19-68284c0455cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:02:17.193724 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da768a60-c8b8-450c-ae19-68284c0455cb None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:02:17.194013 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da768a60-c8b8-450c-ae19-68284c0455cb None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:02:17.194293 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da768a60-c8b8-450c-ae19-68284c0455cb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:02:17.204820 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da768a60-c8b8-450c-ae19-68284c0455cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 16:02:17.209577 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-da768a60-c8b8-450c-ae19-68284c0455cb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 16:02:17.209577 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:02:17.209577 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:02:17.209577 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:02:17.209577 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:02:17.209577 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:02:17.209577 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:02:17.209577 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:02:17.209577 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:02:17.209577 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:02:17.209577 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:02:17.209577 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:02:17.209577 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 16:02:17.209577 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 16:02:17.209577 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 16:02:17.209577 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 16:02:17.212090 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 16:02:17.212090 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 16:02:17.212090 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 16:02:17.212090 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 16:02:17.212090 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:02:17.212090 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:02:17.212090 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:02:17.212090 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:02:17.212090 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:02:17.212090 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:02:17.212090 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:02:17.212090 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:02:17.212090 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:02:17.212090 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:02:17.212090 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:02:17.212090 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:02:17.213073 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:02:17.213073 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:02:17.213073 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:02:17.213073 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:02:17.213073 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 16:02:17.213073 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 16:02:17.213073 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 16:02:17.213073 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 16:02:17.213073 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 16:02:17.213073 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 16:02:17.213073 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 16:02:17.213073 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:02:17.213073 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 596/1188] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 19 16:02:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:02:17.217153 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfeeeee-746b-482c-8765-c0c173d2bfab None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:02:17.320333 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:02:17.324351 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfeeeee-746b-482c-8765-c0c173d2bfab None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:02:17.358737 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9147e304-552d-469e-a095-6dd42bc054ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:02:17.359335 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9147e304-552d-469e-a095-6dd42bc054ad None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:02:17.359607 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9147e304-552d-469e-a095-6dd42bc054ad None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:02:17.359875 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9147e304-552d-469e-a095-6dd42bc054ad None admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:02:17.360522 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9147e304-552d-469e-a095-6dd42bc054ad None admin] RBAC: Adding query filter params (name=admin) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:02:17.360820 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9147e304-552d-469e-a095-6dd42bc054ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:02:17.362170 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9147e304-552d-469e-a095-6dd42bc054ad None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:02:17.371938 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 593/1189] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 16:02:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:02:17.378075 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da768a60-c8b8-450c-ae19-68284c0455cb None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:02:17.438588 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:02:17.486591 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da768a60-c8b8-450c-ae19-68284c0455cb None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:02:17.522070 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2d67b4-3b50-4831-af9c-6e6f0c44b28b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:02:17.522720 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d67b4-3b50-4831-af9c-6e6f0c44b28b None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:02:17.522997 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d67b4-3b50-4831-af9c-6e6f0c44b28b None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:02:17.523268 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d67b4-3b50-4831-af9c-6e6f0c44b28b None admin] PATH_INFO: `/v3/users/7eb3fa89ce11499d9ba9e4bb1afeb26f` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:02:17.524399 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2d67b4-3b50-4831-af9c-6e6f0c44b28b None admin] RBAC: Authorizing `identity:get_user(user_id=7eb3fa89ce11499d9ba9e4bb1afeb26f)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:02:17.526352 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2d67b4-3b50-4831-af9c-6e6f0c44b28b None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:02:17.527964 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 597/1190] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 19 16:02:17 2022] GET /identity/v3/users/7eb3fa89ce11499d9ba9e4bb1afeb26f => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:02:17.534175 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9147e304-552d-469e-a095-6dd42bc054ad None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:02:17.633670 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9147e304-552d-469e-a095-6dd42bc054ad None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:02:17.668660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585499c8-f661-403e-86ff-9b460ac54e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:02:17.669421 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585499c8-f661-403e-86ff-9b460ac54e8e None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:02:17.669809 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585499c8-f661-403e-86ff-9b460ac54e8e None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:02:17.670222 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585499c8-f661-403e-86ff-9b460ac54e8e None admin] PATH_INFO: `/v3/projects/1cd9310705b540dbb786b4f99fde23d7` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:02:17.671711 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585499c8-f661-403e-86ff-9b460ac54e8e None admin] RBAC: Authorizing `identity:get_project(project_id=1cd9310705b540dbb786b4f99fde23d7)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:02:17.673522 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585499c8-f661-403e-86ff-9b460ac54e8e None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:02:17.675828 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 594/1191] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 19 16:02:17 2022] GET /identity/v3/projects/1cd9310705b540dbb786b4f99fde23d7 => generated 361 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:02:17.683623 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2d67b4-3b50-4831-af9c-6e6f0c44b28b None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:02:17.807524 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2d67b4-3b50-4831-af9c-6e6f0c44b28b None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:02:17.845150 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:02:17.854652 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b2a41a-cd78-4e1b-8314-ad31cd78491b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:02:17.855315 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2a41a-cd78-4e1b-8314-ad31cd78491b None admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:02:17.855597 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2a41a-cd78-4e1b-8314-ad31cd78491b None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:02:17.855876 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2a41a-cd78-4e1b-8314-ad31cd78491b None admin] PATH_INFO: `/v3/projects/1cd9310705b540dbb786b4f99fde23d7/users/7eb3fa89ce11499d9ba9e4bb1afeb26f/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:02:17.858271 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b2a41a-cd78-4e1b-8314-ad31cd78491b None admin] RBAC: Authorizing `identity:create_grant(project_id=1cd9310705b540dbb786b4f99fde23d7, user_id=7eb3fa89ce11499d9ba9e4bb1afeb26f, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:02:17.860067 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b2a41a-cd78-4e1b-8314-ad31cd78491b None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:02:17.875117 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 598/1192] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Sep 19 16:02:17 2022] PUT /identity/v3/projects/1cd9310705b540dbb786b4f99fde23d7/users/7eb3fa89ce11499d9ba9e4bb1afeb26f/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:02:20.074308 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d2c163-5229-4e22-a909-6944b6f61622 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:02:20.074904 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d2c163-5229-4e22-a909-6944b6f61622 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:02:20.074904 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d2c163-5229-4e22-a909-6944b6f61622 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:02:20.076117 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 595/1193] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:02:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:02:20.081660 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09df93-9a66-404d-9f2f-1f3fe5688258 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:02:20.082145 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09df93-9a66-404d-9f2f-1f3fe5688258 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:02:20.082284 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09df93-9a66-404d-9f2f-1f3fe5688258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:02:20.100611 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-fe09df93-9a66-404d-9f2f-1f3fe5688258 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:02:20.101820 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-fe09df93-9a66-404d-9f2f-1f3fe5688258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:02:20.180223 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 599/1194] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:02:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 16:02:20.448281 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d2c163-5229-4e22-a909-6944b6f61622 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:02:20.480001 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:02:20.558049 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d2c163-5229-4e22-a909-6944b6f61622 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:02:20.593146 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e2ef38-161a-4cda-8976-040b10257a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:02:20.593774 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e2ef38-161a-4cda-8976-040b10257a3f None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:02:20.594050 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e2ef38-161a-4cda-8976-040b10257a3f None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:02:20.594320 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e2ef38-161a-4cda-8976-040b10257a3f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:02:20.605081 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e2ef38-161a-4cda-8976-040b10257a3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 16:02:20.609983 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-00e2ef38-161a-4cda-8976-040b10257a3f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 16:02:20.609983 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:02:20.609983 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:02:20.609983 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:02:20.609983 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:02:20.609983 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:02:20.609983 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:02:20.609983 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:02:20.609983 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:02:20.609983 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:02:20.609983 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:02:20.609983 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:02:20.609983 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 16:02:20.609983 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 16:02:20.609983 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 16:02:20.609983 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 16:02:20.611275 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 16:02:20.611275 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 16:02:20.611275 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 16:02:20.611275 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 16:02:20.611275 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:02:20.611275 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:02:20.611275 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:02:20.611275 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:02:20.611275 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:02:20.611275 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:02:20.611275 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:02:20.611275 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:02:20.611275 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:02:20.611275 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:02:20.611275 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:02:20.611275 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:02:20.612536 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:02:20.612536 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:02:20.612536 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:02:20.612536 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:02:20.612536 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 16:02:20.612536 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 16:02:20.612536 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 16:02:20.612536 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 16:02:20.612536 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 16:02:20.612536 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 16:02:20.612536 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 16:02:20.612536 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:02:20.612536 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 596/1195] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 19 16:02:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:02:20.617376 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe09df93-9a66-404d-9f2f-1f3fe5688258 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:02:20.723225 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe09df93-9a66-404d-9f2f-1f3fe5688258 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:02:20.747221 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:02:20.765828 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cc4539-2bc4-4f63-b40c-c5cfd1b97076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:02:20.766472 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc4539-2bc4-4f63-b40c-c5cfd1b97076 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:02:20.766745 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc4539-2bc4-4f63-b40c-c5cfd1b97076 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:02:20.767017 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc4539-2bc4-4f63-b40c-c5cfd1b97076 None admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:02:20.767678 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cc4539-2bc4-4f63-b40c-c5cfd1b97076 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:02:20.767979 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cc4539-2bc4-4f63-b40c-c5cfd1b97076 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:02:20.769339 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cc4539-2bc4-4f63-b40c-c5cfd1b97076 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:02:20.778447 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 600/1196] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 16:02:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:02:20.784878 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e2ef38-161a-4cda-8976-040b10257a3f None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:02:20.892253 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:02:20.907006 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e2ef38-161a-4cda-8976-040b10257a3f None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:02:20.951259 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a831db67-bc7d-4c49-9b36-0926d990c7cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:02:20.951977 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a831db67-bc7d-4c49-9b36-0926d990c7cb None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:02:20.952256 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a831db67-bc7d-4c49-9b36-0926d990c7cb None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:02:20.952585 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a831db67-bc7d-4c49-9b36-0926d990c7cb None admin] PATH_INFO: `/v3/users/7eb3fa89ce11499d9ba9e4bb1afeb26f` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:02:20.954085 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a831db67-bc7d-4c49-9b36-0926d990c7cb None admin] RBAC: Authorizing `identity:get_user(user_id=7eb3fa89ce11499d9ba9e4bb1afeb26f)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:02:20.956083 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a831db67-bc7d-4c49-9b36-0926d990c7cb None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:02:20.958723 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 597/1197] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 19 16:02:20 2022] GET /identity/v3/users/7eb3fa89ce11499d9ba9e4bb1afeb26f => generated 311 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:02:20.965314 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cc4539-2bc4-4f63-b40c-c5cfd1b97076 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:02:21.070051 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cc4539-2bc4-4f63-b40c-c5cfd1b97076 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:02:21.104148 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47174983-131f-4062-aa64-ec760eb51335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:02:21.104834 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47174983-131f-4062-aa64-ec760eb51335 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:02:21.105019 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47174983-131f-4062-aa64-ec760eb51335 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:02:21.105291 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47174983-131f-4062-aa64-ec760eb51335 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:02:21.155522 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:02:21.176497 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47174983-131f-4062-aa64-ec760eb51335 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:02:21.178054 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47174983-131f-4062-aa64-ec760eb51335 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:02:21.283176 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 601/1198] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 19 16:02:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 19 16:02:21.289322 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a831db67-bc7d-4c49-9b36-0926d990c7cb None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:02:21.395218 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a831db67-bc7d-4c49-9b36-0926d990c7cb None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:02:21.433392 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09314889-7389-4939-a476-72c3beb7b4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:02:21.434096 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09314889-7389-4939-a476-72c3beb7b4c4 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:02:21.434555 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09314889-7389-4939-a476-72c3beb7b4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:02:21.434751 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09314889-7389-4939-a476-72c3beb7b4c4 None admin] PATH_INFO: `/v3/projects/1cd9310705b540dbb786b4f99fde23d7` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:02:21.435896 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09314889-7389-4939-a476-72c3beb7b4c4 None admin] RBAC: Authorizing `identity:get_project(project_id=1cd9310705b540dbb786b4f99fde23d7)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:02:21.437307 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09314889-7389-4939-a476-72c3beb7b4c4 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:02:21.438969 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 598/1199] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 19 16:02:21 2022] GET /identity/v3/projects/1cd9310705b540dbb786b4f99fde23d7 => generated 361 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:02:21.445620 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47174983-131f-4062-aa64-ec760eb51335 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:02:21.535479 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:02:21.556132 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47174983-131f-4062-aa64-ec760eb51335 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:02:21.594979 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e071bfe-4e0c-4bbb-9d71-f67db37fdd97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:02:21.595588 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e071bfe-4e0c-4bbb-9d71-f67db37fdd97 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:02:21.595870 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e071bfe-4e0c-4bbb-9d71-f67db37fdd97 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:02:21.596149 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e071bfe-4e0c-4bbb-9d71-f67db37fdd97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:02:21.596875 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e071bfe-4e0c-4bbb-9d71-f67db37fdd97 None admin] RBAC: Adding query filter params (role.id=63d2d197bfd6497583e94bfb065ce29b, scope.project.id=1cd9310705b540dbb786b4f99fde23d7, user.id=7eb3fa89ce11499d9ba9e4bb1afeb26f) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:02:21.597185 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e071bfe-4e0c-4bbb-9d71-f67db37fdd97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:02:21.598640 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e071bfe-4e0c-4bbb-9d71-f67db37fdd97 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:02:21.605674 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 602/1200] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 19 16:02:21 2022] GET /identity/v3/role_assignments?user.id=7eb3fa89ce11499d9ba9e4bb1afeb26f&scope.project.id=1cd9310705b540dbb786b4f99fde23d7&role.id=63d2d197bfd6497583e94bfb065ce29b => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:02:31.428015 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=88 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:02:31.601911 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=28 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:03:18.891060 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2efa583-6969-428a-ae27-9f1c7a1d0690 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:18.891653 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2efa583-6969-428a-ae27-9f1c7a1d0690 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:18.891757 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2efa583-6969-428a-ae27-9f1c7a1d0690 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:18.893462 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 599/1201] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:03:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 16:03:18.899695 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e7610-bd25-4619-91cd-58c45d580bed None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:18.900299 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e7610-bd25-4619-91cd-58c45d580bed None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:18.900299 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e7610-bd25-4619-91cd-58c45d580bed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:18.928976 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-b43e7610-bd25-4619-91cd-58c45d580bed None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:03:18.931323 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-b43e7610-bd25-4619-91cd-58c45d580bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:03:19.024965 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 603/1202] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:03:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 16:03:19.356383 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2efa583-6969-428a-ae27-9f1c7a1d0690 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:19.531523 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2efa583-6969-428a-ae27-9f1c7a1d0690 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:19.575972 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288806a8-5669-4c66-858f-bee840dcd7b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:19.576888 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288806a8-5669-4c66-858f-bee840dcd7b1 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:19.577320 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288806a8-5669-4c66-858f-bee840dcd7b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:19.577728 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288806a8-5669-4c66-858f-bee840dcd7b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:19.591761 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288806a8-5669-4c66-858f-bee840dcd7b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:19.597506 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288806a8-5669-4c66-858f-bee840dcd7b1 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:19.615162 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-288806a8-5669-4c66-858f-bee840dcd7b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:03:19.615162 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:03:19.615162 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:03:19.615162 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:03:19.615162 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:03:19.615162 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:03:19.615162 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:03:19.615162 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:03:19.615162 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:03:19.615162 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:03:19.615162 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:03:19.615162 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:03:19.615162 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:03:19.615162 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:03:19.615162 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:03:19.615162 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:03:19.616912 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:03:19.616912 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:03:19.616912 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:03:19.616912 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:03:19.616912 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:03:19.616912 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:03:19.616912 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:03:19.616912 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:03:19.616912 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:03:19.616912 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:03:19.616912 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:03:19.616912 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:03:19.616912 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:03:19.616912 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:03:19.616912 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:03:19.616912 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:03:19.616912 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:03:19.618039 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:03:19.618039 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:03:19.618039 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:03:19.618039 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 600/1203] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:03:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:03:19.624567 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43e7610-bd25-4619-91cd-58c45d580bed None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:19.823121 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43e7610-bd25-4619-91cd-58c45d580bed None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:19.873514 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:03:19.881072 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c47825-b39b-41c8-be74-5644af53a901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:19.881784 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c47825-b39b-41c8-be74-5644af53a901 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:19.882261 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c47825-b39b-41c8-be74-5644af53a901 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:19.882596 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c47825-b39b-41c8-be74-5644af53a901 None admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:19.883973 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c47825-b39b-41c8-be74-5644af53a901 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:03:19.884352 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c47825-b39b-41c8-be74-5644af53a901 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:19.886291 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c47825-b39b-41c8-be74-5644af53a901 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:19.900803 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 604/1204] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:03:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:03:19.907040 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288806a8-5669-4c66-858f-bee840dcd7b1 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:19.976144 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:03:20.045389 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288806a8-5669-4c66-858f-bee840dcd7b1 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:20.092172 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541a0fbc-5571-4e66-a8d7-c9cda433216d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:20.092742 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541a0fbc-5571-4e66-a8d7-c9cda433216d None admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:20.093048 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541a0fbc-5571-4e66-a8d7-c9cda433216d None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:20.093321 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541a0fbc-5571-4e66-a8d7-c9cda433216d None admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:20.094041 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541a0fbc-5571-4e66-a8d7-c9cda433216d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:20.095562 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541a0fbc-5571-4e66-a8d7-c9cda433216d None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:20.129301 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 601/1205] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Sep 19 16:03:19 2022] POST /identity/v3/users => generated 251 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:03:22.634762 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a30e65-6d2f-4187-9450-5cff6cecfe36 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:22.635331 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a30e65-6d2f-4187-9450-5cff6cecfe36 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:22.635331 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a30e65-6d2f-4187-9450-5cff6cecfe36 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:22.637379 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 605/1206] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:03:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:03:22.645522 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24c3b86-8793-4de0-897e-a5f72c4d44bf None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:22.646226 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24c3b86-8793-4de0-897e-a5f72c4d44bf None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:22.646226 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24c3b86-8793-4de0-897e-a5f72c4d44bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:22.670401 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-a24c3b86-8793-4de0-897e-a5f72c4d44bf None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:03:22.671787 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-a24c3b86-8793-4de0-897e-a5f72c4d44bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:03:22.770423 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 602/1207] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:03:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 16:03:23.107763 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a30e65-6d2f-4187-9450-5cff6cecfe36 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:23.264475 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a30e65-6d2f-4187-9450-5cff6cecfe36 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:23.321219 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052e2b11-8a10-4e74-b58a-881217780365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:23.322280 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052e2b11-8a10-4e74-b58a-881217780365 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:23.322777 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052e2b11-8a10-4e74-b58a-881217780365 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:23.323396 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052e2b11-8a10-4e74-b58a-881217780365 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:23.341661 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052e2b11-8a10-4e74-b58a-881217780365 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 16:03:23.349742 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-052e2b11-8a10-4e74-b58a-881217780365 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 16:03:23.349742 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:03:23.349742 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:03:23.349742 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:03:23.349742 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:03:23.349742 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:03:23.349742 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:03:23.349742 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:03:23.349742 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:03:23.349742 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:03:23.349742 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:03:23.349742 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:03:23.349742 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 16:03:23.349742 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 16:03:23.349742 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 16:03:23.349742 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 16:03:23.351680 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 16:03:23.351680 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 16:03:23.351680 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 16:03:23.351680 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 16:03:23.351680 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:03:23.351680 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:03:23.351680 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:03:23.351680 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:03:23.351680 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:03:23.351680 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:03:23.351680 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:03:23.351680 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:03:23.351680 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:03:23.351680 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:03:23.351680 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:03:23.351680 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:03:23.352805 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:03:23.352805 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:03:23.352805 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:03:23.352805 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:03:23.352805 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 16:03:23.352805 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 16:03:23.352805 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 16:03:23.352805 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 16:03:23.352805 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 16:03:23.352805 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 16:03:23.352805 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 16:03:23.352805 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:03:23.352805 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 606/1208] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Sep 19 16:03:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:03:23.361422 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24c3b86-8793-4de0-897e-a5f72c4d44bf None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:23.413662 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:03:23.503708 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24c3b86-8793-4de0-897e-a5f72c4d44bf None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:23.553989 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200d3f55-9ffa-43f6-a598-2ba3d5aff6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:23.554747 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d3f55-9ffa-43f6-a598-2ba3d5aff6b4 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:23.556190 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d3f55-9ffa-43f6-a598-2ba3d5aff6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:23.556190 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d3f55-9ffa-43f6-a598-2ba3d5aff6b4 None admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:23.556190 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200d3f55-9ffa-43f6-a598-2ba3d5aff6b4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:03:23.556543 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200d3f55-9ffa-43f6-a598-2ba3d5aff6b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:23.558417 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200d3f55-9ffa-43f6-a598-2ba3d5aff6b4 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:23.568919 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 603/1209] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Sep 19 16:03:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:03:23.577854 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052e2b11-8a10-4e74-b58a-881217780365 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:23.655031 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:03:23.733738 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052e2b11-8a10-4e74-b58a-881217780365 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:23.789284 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a9c475-19e3-412b-a4a4-e41a05e4d815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:23.790148 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a9c475-19e3-412b-a4a4-e41a05e4d815 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:23.790659 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a9c475-19e3-412b-a4a4-e41a05e4d815 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:23.791327 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a9c475-19e3-412b-a4a4-e41a05e4d815 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:23.807283 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a9c475-19e3-412b-a4a4-e41a05e4d815 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:23.808872 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a9c475-19e3-412b-a4a4-e41a05e4d815 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:23.835839 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-f6a9c475-19e3-412b-a4a4-e41a05e4d815 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:03:23.835839 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:03:23.835839 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:03:23.835839 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:03:23.835839 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:03:23.835839 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:03:23.835839 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:03:23.835839 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:03:23.835839 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:03:23.835839 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:03:23.835839 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:03:23.835839 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:03:23.835839 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:03:23.835839 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:03:23.835839 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:03:23.835839 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:03:23.837070 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:03:23.837070 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:03:23.837070 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:03:23.837070 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:03:23.837070 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:03:23.837070 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:03:23.837070 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:03:23.837070 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:03:23.837070 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:03:23.837070 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:03:23.837070 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:03:23.837070 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:03:23.837070 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:03:23.837070 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:03:23.837070 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:03:23.837070 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:03:23.837070 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:03:23.838731 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:03:23.838731 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:03:23.838731 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:03:23.838731 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 607/1210] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:03:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:03:23.844259 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200d3f55-9ffa-43f6-a598-2ba3d5aff6b4 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:23.999042 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:03:24.005933 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200d3f55-9ffa-43f6-a598-2ba3d5aff6b4 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:24.058309 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30fe11a-0486-43e5-a222-2d78e0e0262b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:24.059311 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30fe11a-0486-43e5-a222-2d78e0e0262b None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:24.059704 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30fe11a-0486-43e5-a222-2d78e0e0262b None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:24.060026 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30fe11a-0486-43e5-a222-2d78e0e0262b None admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:24.061072 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30fe11a-0486-43e5-a222-2d78e0e0262b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:03:24.061531 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30fe11a-0486-43e5-a222-2d78e0e0262b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:24.063472 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30fe11a-0486-43e5-a222-2d78e0e0262b None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:24.081033 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 604/1211] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:03:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:03:24.088197 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a9c475-19e3-412b-a4a4-e41a05e4d815 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:24.231937 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a9c475-19e3-412b-a4a4-e41a05e4d815 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:24.234405 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:03:24.279594 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78281722-ebca-4ed5-b82a-69e5ef1e8989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:24.280541 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78281722-ebca-4ed5-b82a-69e5ef1e8989 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:24.281093 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78281722-ebca-4ed5-b82a-69e5ef1e8989 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:24.281614 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78281722-ebca-4ed5-b82a-69e5ef1e8989 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:24.291459 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78281722-ebca-4ed5-b82a-69e5ef1e8989 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:24.292907 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78281722-ebca-4ed5-b82a-69e5ef1e8989 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:24.308979 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-78281722-ebca-4ed5-b82a-69e5ef1e8989 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 19 16:03:24.308979 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:03:24.308979 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:03:24.308979 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:03:24.308979 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:03:24.308979 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:03:24.308979 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:03:24.308979 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:03:24.308979 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:03:24.308979 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:03:24.308979 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:03:24.308979 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:03:24.308979 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 16:03:24.308979 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 16:03:24.308979 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 16:03:24.308979 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 16:03:24.310249 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:03:24.310249 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:03:24.310249 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 16:03:24.310249 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:03:24.310249 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 16:03:24.310249 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:03:24.310249 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:03:24.310249 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:03:24.310249 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:03:24.310249 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:03:24.310249 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:03:24.310249 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:03:24.310249 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:03:24.310249 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:03:24.310249 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:03:24.310249 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:03:24.310249 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:03:24.310249 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:03:24.311254 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:03:24.311254 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:03:24.311254 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 16:03:24.311254 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 16:03:24.311254 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 16:03:24.311254 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 16:03:24.311254 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 16:03:24.311254 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 16:03:24.311254 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 19 16:03:24.311254 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:03:24.311254 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 608/1212] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:03:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 16:03:24.321314 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30fe11a-0486-43e5-a222-2d78e0e0262b None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:24.463150 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30fe11a-0486-43e5-a222-2d78e0e0262b None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:24.513175 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67426649-3d89-4a85-8902-31ae37af4727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:24.514110 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67426649-3d89-4a85-8902-31ae37af4727 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:24.514478 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67426649-3d89-4a85-8902-31ae37af4727 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:24.514846 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67426649-3d89-4a85-8902-31ae37af4727 None admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:24.515960 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67426649-3d89-4a85-8902-31ae37af4727 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:03:24.516404 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67426649-3d89-4a85-8902-31ae37af4727 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:24.518508 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67426649-3d89-4a85-8902-31ae37af4727 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:24.550299 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 605/1213] 10.222.0.57 () {56 vars in 1143 bytes} [Mon Sep 19 16:03:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:03:24.552731 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78281722-ebca-4ed5-b82a-69e5ef1e8989 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:24.686994 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78281722-ebca-4ed5-b82a-69e5ef1e8989 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:24.732128 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa3834a-e5e9-4ba2-9a7a-30d4da32d484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:24.732709 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa3834a-e5e9-4ba2-9a7a-30d4da32d484 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:24.732986 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa3834a-e5e9-4ba2-9a7a-30d4da32d484 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:24.733265 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa3834a-e5e9-4ba2-9a7a-30d4da32d484 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:24.779350 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:03:24.828988 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa3834a-e5e9-4ba2-9a7a-30d4da32d484 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:24.830430 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa3834a-e5e9-4ba2-9a7a-30d4da32d484 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:24.991062 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 609/1214] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 19 16:03:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 16:03:24.996277 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67426649-3d89-4a85-8902-31ae37af4727 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:25.033109 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:03:25.205514 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67426649-3d89-4a85-8902-31ae37af4727 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:25.258460 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9c64d9-681f-4da0-9f04-3772ca6960d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:25.259222 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c64d9-681f-4da0-9f04-3772ca6960d6 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:25.259481 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c64d9-681f-4da0-9f04-3772ca6960d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:25.259761 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c64d9-681f-4da0-9f04-3772ca6960d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:25.278486 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9c64d9-681f-4da0-9f04-3772ca6960d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:25.281550 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9c64d9-681f-4da0-9f04-3772ca6960d6 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:25.301642 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-fc9c64d9-681f-4da0-9f04-3772ca6960d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:03:25.301642 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:03:25.301642 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:03:25.301642 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:03:25.301642 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:03:25.301642 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:03:25.301642 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:03:25.301642 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:03:25.301642 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:03:25.301642 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:03:25.301642 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:03:25.301642 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:03:25.301642 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:03:25.301642 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:03:25.301642 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:03:25.301642 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:03:25.302945 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:03:25.302945 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:03:25.302945 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:03:25.302945 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:03:25.302945 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:03:25.302945 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:03:25.302945 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:03:25.302945 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:03:25.302945 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:03:25.302945 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:03:25.302945 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:03:25.302945 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:03:25.302945 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:03:25.302945 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:03:25.302945 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:03:25.302945 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:03:25.302945 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:03:25.304089 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:03:25.304089 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:03:25.304089 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:03:25.304089 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 606/1215] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:03:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:03:25.309927 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa3834a-e5e9-4ba2-9a7a-30d4da32d484 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:25.406941 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:03:25.474929 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa3834a-e5e9-4ba2-9a7a-30d4da32d484 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:25.526353 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84d8cd9-0254-414c-abb5-a977b178911f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:25.527046 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d8cd9-0254-414c-abb5-a977b178911f None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:25.527375 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d8cd9-0254-414c-abb5-a977b178911f None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:25.527726 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d8cd9-0254-414c-abb5-a977b178911f None admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:25.528440 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84d8cd9-0254-414c-abb5-a977b178911f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:03:25.528787 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84d8cd9-0254-414c-abb5-a977b178911f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:25.530320 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84d8cd9-0254-414c-abb5-a977b178911f None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:25.552241 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 610/1216] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:03:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:03:25.553916 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9c64d9-681f-4da0-9f04-3772ca6960d6 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:25.645960 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:03:25.686360 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9c64d9-681f-4da0-9f04-3772ca6960d6 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:25.728769 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d81fc-ca4c-4ca3-911c-c6ff3316dfe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:25.729364 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d81fc-ca4c-4ca3-911c-c6ff3316dfe8 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:25.729734 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d81fc-ca4c-4ca3-911c-c6ff3316dfe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:25.729996 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d81fc-ca4c-4ca3-911c-c6ff3316dfe8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:25.737469 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139d81fc-ca4c-4ca3-911c-c6ff3316dfe8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:25.738958 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139d81fc-ca4c-4ca3-911c-c6ff3316dfe8 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:25.756859 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-139d81fc-ca4c-4ca3-911c-c6ff3316dfe8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:03:25.756859 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:03:25.756859 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:03:25.756859 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:03:25.756859 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:03:25.756859 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:03:25.756859 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:03:25.756859 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:03:25.756859 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:03:25.756859 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:03:25.756859 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:03:25.756859 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:03:25.756859 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 16:03:25.756859 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 16:03:25.756859 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 16:03:25.758440 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 16:03:25.758440 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:03:25.758440 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:03:25.758440 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:03:25.758440 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:03:25.758440 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:03:25.758440 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:03:25.758440 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:03:25.758440 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:03:25.758440 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:03:25.758440 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:03:25.758440 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:03:25.758440 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:03:25.758440 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:03:25.758440 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:03:25.758440 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:03:25.758440 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:03:25.759465 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 16:03:25.759465 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 16:03:25.759465 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 16:03:25.759465 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 16:03:25.759465 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 16:03:25.759465 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 16:03:25.759465 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:03:25.759465 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:03:25.759465 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 607/1217] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 19 16:03:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:03:25.768438 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84d8cd9-0254-414c-abb5-a977b178911f None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:25.902876 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84d8cd9-0254-414c-abb5-a977b178911f None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:25.934203 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:03:25.956384 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f8bf0e-2cef-4c0e-9367-e37144d317e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:25.957355 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f8bf0e-2cef-4c0e-9367-e37144d317e5 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:25.957823 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f8bf0e-2cef-4c0e-9367-e37144d317e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:25.958278 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f8bf0e-2cef-4c0e-9367-e37144d317e5 None admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:25.959405 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f8bf0e-2cef-4c0e-9367-e37144d317e5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:03:25.959908 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f8bf0e-2cef-4c0e-9367-e37144d317e5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:25.962199 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f8bf0e-2cef-4c0e-9367-e37144d317e5 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:25.981175 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 611/1218] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 19 16:03:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:03:25.990153 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d81fc-ca4c-4ca3-911c-c6ff3316dfe8 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:26.135856 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d81fc-ca4c-4ca3-911c-c6ff3316dfe8 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:26.173309 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:03:26.181506 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eacd28-b1b6-4319-86db-60a8b2edeaaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:26.182248 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eacd28-b1b6-4319-86db-60a8b2edeaaa None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:26.182606 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eacd28-b1b6-4319-86db-60a8b2edeaaa None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:26.182922 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eacd28-b1b6-4319-86db-60a8b2edeaaa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:26.183775 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eacd28-b1b6-4319-86db-60a8b2edeaaa None admin] RBAC: Adding query filter params (role.id=1e475e6a170a4dbabc4ce8348eeb30b9, scope.project.id=be03e5300b3343169f35c643c93d1e06, user.id=1f1a985495ed48a4acc48e2e6fc7f977) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:03:26.184079 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eacd28-b1b6-4319-86db-60a8b2edeaaa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:26.185529 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eacd28-b1b6-4319-86db-60a8b2edeaaa None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:26.191601 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 608/1219] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 19 16:03:25 2022] GET /identity/v3/role_assignments?user.id=1f1a985495ed48a4acc48e2e6fc7f977&scope.project.id=be03e5300b3343169f35c643c93d1e06&role.id=1e475e6a170a4dbabc4ce8348eeb30b9 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:03:28.444126 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f71e6f-dc92-4690-a6b9-a4f584befd25 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:28.444676 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f71e6f-dc92-4690-a6b9-a4f584befd25 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:28.444676 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f71e6f-dc92-4690-a6b9-a4f584befd25 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:28.446847 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 612/1220] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:03:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:03:28.452089 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be55a32-4fdc-4195-b0bb-e9ed5f0cc554 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:28.452575 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be55a32-4fdc-4195-b0bb-e9ed5f0cc554 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:28.452662 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be55a32-4fdc-4195-b0bb-e9ed5f0cc554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:28.471579 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-3be55a32-4fdc-4195-b0bb-e9ed5f0cc554 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:03:28.472775 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-3be55a32-4fdc-4195-b0bb-e9ed5f0cc554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:03:28.541408 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 609/1221] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:03:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 16:03:28.830244 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f71e6f-dc92-4690-a6b9-a4f584befd25 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:28.944130 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f71e6f-dc92-4690-a6b9-a4f584befd25 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:28.991111 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5df954-d269-4a65-8add-51b5e6876e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:28.991793 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5df954-d269-4a65-8add-51b5e6876e5f None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:28.992057 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5df954-d269-4a65-8add-51b5e6876e5f None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:28.992344 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5df954-d269-4a65-8add-51b5e6876e5f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:29.004547 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5df954-d269-4a65-8add-51b5e6876e5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 16:03:29.011625 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-2c5df954-d269-4a65-8add-51b5e6876e5f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 16:03:29.011625 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:03:29.011625 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:03:29.011625 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:03:29.011625 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:03:29.011625 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:03:29.011625 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:03:29.011625 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:03:29.011625 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:03:29.011625 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:03:29.011625 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:03:29.011625 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:03:29.011625 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 16:03:29.011625 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 16:03:29.011625 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 16:03:29.011625 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 16:03:29.013024 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 16:03:29.013024 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 16:03:29.013024 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 16:03:29.013024 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 16:03:29.013024 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:03:29.013024 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:03:29.013024 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:03:29.013024 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:03:29.013024 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:03:29.013024 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:03:29.013024 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:03:29.013024 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:03:29.013024 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:03:29.013024 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:03:29.013024 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:03:29.013024 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:03:29.014405 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:03:29.014405 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:03:29.014405 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:03:29.014405 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:03:29.014405 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 16:03:29.014405 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 16:03:29.014405 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 16:03:29.014405 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 16:03:29.014405 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 16:03:29.014405 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 16:03:29.014405 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 16:03:29.014405 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:03:29.021776 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 613/1222] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Sep 19 16:03:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:03:29.022713 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be55a32-4fdc-4195-b0bb-e9ed5f0cc554 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:29.138409 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be55a32-4fdc-4195-b0bb-e9ed5f0cc554 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:29.166556 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:03:29.179480 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20291bb6-221a-4994-8d0c-7f80690353fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:29.180231 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20291bb6-221a-4994-8d0c-7f80690353fd None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:29.180532 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20291bb6-221a-4994-8d0c-7f80690353fd None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:29.180815 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20291bb6-221a-4994-8d0c-7f80690353fd None admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:29.181562 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20291bb6-221a-4994-8d0c-7f80690353fd None admin] RBAC: Adding query filter params (name=service) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:03:29.181925 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20291bb6-221a-4994-8d0c-7f80690353fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:29.184081 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20291bb6-221a-4994-8d0c-7f80690353fd None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:29.193650 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 610/1223] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Sep 19 16:03:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:03:29.200513 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5df954-d269-4a65-8add-51b5e6876e5f None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:29.252402 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:03:29.328391 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5df954-d269-4a65-8add-51b5e6876e5f None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:29.370273 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eda217c-1754-4560-9374-20dc1fc0e162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:29.370983 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda217c-1754-4560-9374-20dc1fc0e162 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:29.371317 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda217c-1754-4560-9374-20dc1fc0e162 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:29.371557 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda217c-1754-4560-9374-20dc1fc0e162 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:29.383885 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eda217c-1754-4560-9374-20dc1fc0e162 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:29.385403 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eda217c-1754-4560-9374-20dc1fc0e162 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:29.406097 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-5eda217c-1754-4560-9374-20dc1fc0e162 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:03:29.406097 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:03:29.406097 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:03:29.406097 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:03:29.406097 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:03:29.406097 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:03:29.406097 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:03:29.406097 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:03:29.406097 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:03:29.406097 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:03:29.406097 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:03:29.406097 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:03:29.406097 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:03:29.406097 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:03:29.406097 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:03:29.406097 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:03:29.407290 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:03:29.407290 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:03:29.407290 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:03:29.407290 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:03:29.407290 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:03:29.407290 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:03:29.407290 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:03:29.407290 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:03:29.407290 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:03:29.407290 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:03:29.407290 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:03:29.407290 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:03:29.407290 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:03:29.407290 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:03:29.407290 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:03:29.407290 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:03:29.407290 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:03:29.408617 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:03:29.408617 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:03:29.408617 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:03:29.408617 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 614/1224] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:03:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:03:29.414391 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20291bb6-221a-4994-8d0c-7f80690353fd None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:29.533375 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20291bb6-221a-4994-8d0c-7f80690353fd None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:29.591355 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afa1922-86c0-4099-af22-718a164971d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:29.592034 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afa1922-86c0-4099-af22-718a164971d6 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:29.592315 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afa1922-86c0-4099-af22-718a164971d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:29.592636 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afa1922-86c0-4099-af22-718a164971d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:29.593295 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afa1922-86c0-4099-af22-718a164971d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:03:29.593607 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afa1922-86c0-4099-af22-718a164971d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:29.595126 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afa1922-86c0-4099-af22-718a164971d6 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:29.608015 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 611/1225] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:03:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:03:29.613940 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eda217c-1754-4560-9374-20dc1fc0e162 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:29.725184 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:03:29.733639 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eda217c-1754-4560-9374-20dc1fc0e162 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:29.770054 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156c0916-cfb8-431d-ac81-6ce9768ebb86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:29.770716 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156c0916-cfb8-431d-ac81-6ce9768ebb86 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:29.770990 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156c0916-cfb8-431d-ac81-6ce9768ebb86 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:29.771263 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156c0916-cfb8-431d-ac81-6ce9768ebb86 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:29.779891 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156c0916-cfb8-431d-ac81-6ce9768ebb86 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:29.781462 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156c0916-cfb8-431d-ac81-6ce9768ebb86 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:29.794581 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-156c0916-cfb8-431d-ac81-6ce9768ebb86 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 19 16:03:29.794581 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:03:29.794581 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:03:29.794581 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:03:29.794581 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:03:29.794581 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:03:29.794581 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:03:29.794581 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:03:29.794581 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:03:29.794581 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:03:29.794581 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:03:29.794581 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:03:29.794581 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 16:03:29.794581 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 16:03:29.794581 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 16:03:29.794581 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 16:03:29.795744 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:03:29.795744 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:03:29.795744 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 16:03:29.795744 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:03:29.795744 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 16:03:29.795744 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:03:29.795744 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:03:29.795744 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:03:29.795744 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:03:29.795744 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:03:29.795744 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:03:29.795744 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:03:29.795744 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:03:29.795744 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:03:29.795744 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:03:29.795744 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:03:29.795744 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:03:29.795744 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:03:29.796909 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:03:29.796909 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:03:29.796909 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 16:03:29.796909 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 16:03:29.796909 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 16:03:29.796909 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 16:03:29.796909 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 16:03:29.796909 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 16:03:29.796909 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 19 16:03:29.796909 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:03:29.796909 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 615/1226] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:03:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:03:29.801828 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afa1922-86c0-4099-af22-718a164971d6 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:29.862649 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:03:29.935803 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afa1922-86c0-4099-af22-718a164971d6 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:29.981783 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f11c8e-a8cf-47d2-a705-ebaffb4813ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:29.982348 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f11c8e-a8cf-47d2-a705-ebaffb4813ae None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:29.982678 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f11c8e-a8cf-47d2-a705-ebaffb4813ae None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:29.982955 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f11c8e-a8cf-47d2-a705-ebaffb4813ae None admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:29.983839 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f11c8e-a8cf-47d2-a705-ebaffb4813ae None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:03:29.984123 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f11c8e-a8cf-47d2-a705-ebaffb4813ae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:29.985546 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f11c8e-a8cf-47d2-a705-ebaffb4813ae None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:30.004787 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 612/1227] 10.222.0.57 () {56 vars in 1143 bytes} [Mon Sep 19 16:03:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:03:30.012703 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156c0916-cfb8-431d-ac81-6ce9768ebb86 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:30.127802 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156c0916-cfb8-431d-ac81-6ce9768ebb86 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:30.165656 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77d2339-42c7-4cc6-bf80-f052b081829d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:30.166292 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d2339-42c7-4cc6-bf80-f052b081829d None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:30.166603 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d2339-42c7-4cc6-bf80-f052b081829d None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:30.166875 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d2339-42c7-4cc6-bf80-f052b081829d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:30.178575 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77d2339-42c7-4cc6-bf80-f052b081829d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:30.180022 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77d2339-42c7-4cc6-bf80-f052b081829d None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:30.194751 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-d77d2339-42c7-4cc6-bf80-f052b081829d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:03:30.194751 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:03:30.194751 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:03:30.194751 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:03:30.194751 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:03:30.194751 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:03:30.194751 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:03:30.194751 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:03:30.194751 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:03:30.194751 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:03:30.194751 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:03:30.194751 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:03:30.194751 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:03:30.194751 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:03:30.194751 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:03:30.194751 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:03:30.196420 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:03:30.196420 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:03:30.196420 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:03:30.196420 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:03:30.196420 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:03:30.196420 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:03:30.196420 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:03:30.196420 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:03:30.196420 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:03:30.196420 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:03:30.196420 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:03:30.196420 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:03:30.196420 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:03:30.196420 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:03:30.196420 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:03:30.196420 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:03:30.196420 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:03:30.197423 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:03:30.197423 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:03:30.197423 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:03:30.197423 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 616/1228] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:03:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:03:30.202661 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f11c8e-a8cf-47d2-a705-ebaffb4813ae None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:30.309389 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f11c8e-a8cf-47d2-a705-ebaffb4813ae None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:30.311961 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:03:30.351119 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb26198f-abd2-45bf-8885-dfa8d9d0965d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:30.351855 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb26198f-abd2-45bf-8885-dfa8d9d0965d None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:30.352214 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb26198f-abd2-45bf-8885-dfa8d9d0965d None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:30.352488 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb26198f-abd2-45bf-8885-dfa8d9d0965d None admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:30.353166 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb26198f-abd2-45bf-8885-dfa8d9d0965d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:03:30.353481 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb26198f-abd2-45bf-8885-dfa8d9d0965d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:30.355016 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb26198f-abd2-45bf-8885-dfa8d9d0965d None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:30.372280 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 613/1229] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:03:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:03:30.373562 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77d2339-42c7-4cc6-bf80-f052b081829d None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:30.384169 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:03:30.482972 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77d2339-42c7-4cc6-bf80-f052b081829d None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:30.520365 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6171ff08-79c2-427f-b204-f2476852e90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:30.521181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6171ff08-79c2-427f-b204-f2476852e90d None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:30.521545 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6171ff08-79c2-427f-b204-f2476852e90d None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:30.522036 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6171ff08-79c2-427f-b204-f2476852e90d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:30.531248 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6171ff08-79c2-427f-b204-f2476852e90d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:30.533145 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6171ff08-79c2-427f-b204-f2476852e90d None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:30.546469 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-6171ff08-79c2-427f-b204-f2476852e90d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:03:30.546469 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:03:30.546469 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:03:30.546469 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:03:30.546469 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:03:30.546469 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:03:30.546469 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:03:30.546469 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:03:30.546469 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:03:30.546469 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:03:30.546469 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:03:30.546469 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:03:30.546469 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 16:03:30.546469 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 16:03:30.546469 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 16:03:30.547622 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 16:03:30.547622 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:03:30.547622 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:03:30.547622 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:03:30.547622 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:03:30.547622 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:03:30.547622 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:03:30.547622 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:03:30.547622 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:03:30.547622 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:03:30.547622 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:03:30.547622 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:03:30.547622 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:03:30.547622 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:03:30.547622 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:03:30.547622 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:03:30.547622 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:03:30.548883 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 16:03:30.548883 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 16:03:30.548883 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 16:03:30.548883 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 16:03:30.548883 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 16:03:30.548883 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 16:03:30.548883 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:03:30.548883 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:03:30.548883 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 617/1230] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 19 16:03:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:03:30.554504 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb26198f-abd2-45bf-8885-dfa8d9d0965d None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:30.672346 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb26198f-abd2-45bf-8885-dfa8d9d0965d None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:30.711598 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a33e81-9d2e-47be-916b-7e1bc9df2251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:30.712222 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a33e81-9d2e-47be-916b-7e1bc9df2251 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:30.712499 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a33e81-9d2e-47be-916b-7e1bc9df2251 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:30.712805 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a33e81-9d2e-47be-916b-7e1bc9df2251 None admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:30.713766 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a33e81-9d2e-47be-916b-7e1bc9df2251 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:03:30.714130 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a33e81-9d2e-47be-916b-7e1bc9df2251 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:30.716615 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a33e81-9d2e-47be-916b-7e1bc9df2251 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:30.731470 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 614/1231] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 19 16:03:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:03:30.738164 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6171ff08-79c2-427f-b204-f2476852e90d None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:30.811870 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:03:30.851308 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6171ff08-79c2-427f-b204-f2476852e90d None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:30.886176 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74170ac8-2f1d-4572-bae2-6eef081ad9b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:30.886875 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74170ac8-2f1d-4572-bae2-6eef081ad9b5 None admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:30.887158 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74170ac8-2f1d-4572-bae2-6eef081ad9b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:30.887433 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74170ac8-2f1d-4572-bae2-6eef081ad9b5 None admin] PATH_INFO: `/v3/projects/be03e5300b3343169f35c643c93d1e06/users/1f1a985495ed48a4acc48e2e6fc7f977/roles/1e475e6a170a4dbabc4ce8348eeb30b9` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:30.904244 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74170ac8-2f1d-4572-bae2-6eef081ad9b5 None admin] RBAC: Authorizing `identity:create_grant(project_id=be03e5300b3343169f35c643c93d1e06, user_id=1f1a985495ed48a4acc48e2e6fc7f977, role_id=1e475e6a170a4dbabc4ce8348eeb30b9)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:30.905706 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74170ac8-2f1d-4572-bae2-6eef081ad9b5 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:30.920210 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 618/1232] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Sep 19 16:03:30 2022] PUT /identity/v3/projects/be03e5300b3343169f35c643c93d1e06/users/1f1a985495ed48a4acc48e2e6fc7f977/roles/1e475e6a170a4dbabc4ce8348eeb30b9 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:03:33.314016 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae1223-beaf-4750-b7ed-9e9d68079ff3 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:33.314934 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae1223-beaf-4750-b7ed-9e9d68079ff3 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:33.314934 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae1223-beaf-4750-b7ed-9e9d68079ff3 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:33.316311 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 615/1233] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:03:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:03:33.322155 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b25b4e0-43ad-495c-a07a-bfb4ec7f2a4a None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:33.322652 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b25b4e0-43ad-495c-a07a-bfb4ec7f2a4a None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:33.322752 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b25b4e0-43ad-495c-a07a-bfb4ec7f2a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:33.347915 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-4b25b4e0-43ad-495c-a07a-bfb4ec7f2a4a None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:03:33.349511 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-4b25b4e0-43ad-495c-a07a-bfb4ec7f2a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:03:33.446098 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 619/1234] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:03:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 16:03:33.740989 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aae1223-beaf-4750-b7ed-9e9d68079ff3 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:33.770575 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:03:33.883478 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aae1223-beaf-4750-b7ed-9e9d68079ff3 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:33.926494 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ae1042-55b3-4a4b-927a-bcd4687fb1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:33.927371 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae1042-55b3-4a4b-927a-bcd4687fb1f4 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:33.927784 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae1042-55b3-4a4b-927a-bcd4687fb1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:33.928074 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae1042-55b3-4a4b-927a-bcd4687fb1f4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:33.940044 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ae1042-55b3-4a4b-927a-bcd4687fb1f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 16:03:33.946315 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-d6ae1042-55b3-4a4b-927a-bcd4687fb1f4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 16:03:33.946315 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:03:33.946315 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:03:33.946315 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:03:33.946315 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:03:33.946315 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:03:33.946315 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:03:33.946315 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:03:33.946315 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:03:33.946315 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:03:33.946315 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:03:33.946315 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:03:33.946315 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 16:03:33.946315 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 16:03:33.946315 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 16:03:33.946315 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 16:03:33.947506 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 16:03:33.947506 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 16:03:33.947506 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 16:03:33.947506 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 16:03:33.947506 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:03:33.947506 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:03:33.947506 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:03:33.947506 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:03:33.947506 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:03:33.947506 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:03:33.947506 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:03:33.947506 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:03:33.947506 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:03:33.947506 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:03:33.947506 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:03:33.947506 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:03:33.948396 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:03:33.948396 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:03:33.948396 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:03:33.948396 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:03:33.948396 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 16:03:33.948396 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 16:03:33.948396 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 16:03:33.948396 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 16:03:33.948396 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 16:03:33.948396 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 16:03:33.948396 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 16:03:33.948396 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:03:33.948396 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 616/1235] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Sep 19 16:03:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:03:33.954681 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b25b4e0-43ad-495c-a07a-bfb4ec7f2a4a None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:34.012100 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:03:34.097332 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b25b4e0-43ad-495c-a07a-bfb4ec7f2a4a None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:34.146827 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db637c4a-7722-43c3-8fa5-834ba7a8701f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:34.147453 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db637c4a-7722-43c3-8fa5-834ba7a8701f None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:34.147731 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db637c4a-7722-43c3-8fa5-834ba7a8701f None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:34.148078 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db637c4a-7722-43c3-8fa5-834ba7a8701f None admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:34.148686 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db637c4a-7722-43c3-8fa5-834ba7a8701f None admin] RBAC: Adding query filter params (name=service) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:03:34.148997 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db637c4a-7722-43c3-8fa5-834ba7a8701f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:34.150499 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db637c4a-7722-43c3-8fa5-834ba7a8701f None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:34.160674 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 620/1236] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Sep 19 16:03:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:03:34.167954 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ae1042-55b3-4a4b-927a-bcd4687fb1f4 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:34.274905 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:03:34.296293 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ae1042-55b3-4a4b-927a-bcd4687fb1f4 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:34.345611 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821133cf-cf90-43ab-b443-b878b4ba1351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:34.346718 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821133cf-cf90-43ab-b443-b878b4ba1351 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:34.346718 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821133cf-cf90-43ab-b443-b878b4ba1351 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:34.346869 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821133cf-cf90-43ab-b443-b878b4ba1351 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:34.360924 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821133cf-cf90-43ab-b443-b878b4ba1351 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:34.362735 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821133cf-cf90-43ab-b443-b878b4ba1351 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:34.380860 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-821133cf-cf90-43ab-b443-b878b4ba1351 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:03:34.380860 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:03:34.380860 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:03:34.380860 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:03:34.380860 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:03:34.380860 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:03:34.380860 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:03:34.380860 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:03:34.380860 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:03:34.380860 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:03:34.380860 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:03:34.380860 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:03:34.380860 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:03:34.380860 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:03:34.380860 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:03:34.380860 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:03:34.382110 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:03:34.382110 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:03:34.382110 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:03:34.382110 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:03:34.382110 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:03:34.382110 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:03:34.382110 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:03:34.382110 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:03:34.382110 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:03:34.382110 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:03:34.382110 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:03:34.382110 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:03:34.382110 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:03:34.382110 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:03:34.382110 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:03:34.382110 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:03:34.382110 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:03:34.383830 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:03:34.383830 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:03:34.383830 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:03:34.383830 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 617/1237] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:03:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 16:03:34.395171 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db637c4a-7722-43c3-8fa5-834ba7a8701f None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:34.567611 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db637c4a-7722-43c3-8fa5-834ba7a8701f None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:34.580334 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:03:34.618337 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec5e789-e465-43d4-b3c4-e530f5f70e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:34.620052 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec5e789-e465-43d4-b3c4-e530f5f70e10 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:34.620378 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec5e789-e465-43d4-b3c4-e530f5f70e10 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:34.620870 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec5e789-e465-43d4-b3c4-e530f5f70e10 None admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:34.621611 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec5e789-e465-43d4-b3c4-e530f5f70e10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:03:34.621828 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec5e789-e465-43d4-b3c4-e530f5f70e10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:34.623241 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec5e789-e465-43d4-b3c4-e530f5f70e10 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:34.636723 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 621/1238] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:03:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:03:34.642273 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821133cf-cf90-43ab-b443-b878b4ba1351 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:34.769137 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821133cf-cf90-43ab-b443-b878b4ba1351 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:34.810845 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed75ae8-9cbe-48cc-bb76-c95e46b87208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:34.811499 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed75ae8-9cbe-48cc-bb76-c95e46b87208 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:34.811786 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed75ae8-9cbe-48cc-bb76-c95e46b87208 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:34.812023 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed75ae8-9cbe-48cc-bb76-c95e46b87208 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:34.815488 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:03:34.823189 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed75ae8-9cbe-48cc-bb76-c95e46b87208 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:34.825470 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed75ae8-9cbe-48cc-bb76-c95e46b87208 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:34.842121 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-aed75ae8-9cbe-48cc-bb76-c95e46b87208 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 19 16:03:34.842121 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:03:34.842121 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:03:34.842121 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:03:34.842121 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:03:34.842121 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:03:34.842121 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:03:34.842121 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:03:34.842121 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:03:34.842121 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:03:34.842121 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:03:34.842121 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:03:34.842121 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 16:03:34.842121 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 16:03:34.842121 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 16:03:34.842121 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 16:03:34.844381 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:03:34.844381 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:03:34.844381 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 16:03:34.844381 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:03:34.844381 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 16:03:34.844381 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:03:34.844381 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:03:34.844381 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:03:34.844381 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:03:34.844381 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:03:34.844381 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:03:34.844381 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:03:34.844381 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:03:34.844381 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:03:34.844381 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:03:34.844381 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:03:34.844381 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:03:34.844381 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:03:34.846314 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:03:34.846314 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:03:34.846314 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 16:03:34.846314 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 16:03:34.846314 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 16:03:34.846314 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 16:03:34.846314 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 16:03:34.846314 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 16:03:34.846314 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 19 16:03:34.846314 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:03:34.846314 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 618/1239] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:03:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:03:34.851053 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec5e789-e465-43d4-b3c4-e530f5f70e10 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:34.988470 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec5e789-e465-43d4-b3c4-e530f5f70e10 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:35.033556 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f1d2ab-00cc-4bb8-ae91-b57ac23a3593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:35.034179 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f1d2ab-00cc-4bb8-ae91-b57ac23a3593 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:35.034480 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f1d2ab-00cc-4bb8-ae91-b57ac23a3593 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:35.034785 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f1d2ab-00cc-4bb8-ae91-b57ac23a3593 None admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:35.035511 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f1d2ab-00cc-4bb8-ae91-b57ac23a3593 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:03:35.035819 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f1d2ab-00cc-4bb8-ae91-b57ac23a3593 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:35.037258 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f1d2ab-00cc-4bb8-ae91-b57ac23a3593 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:35.061316 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 622/1240] 10.222.0.57 () {56 vars in 1143 bytes} [Mon Sep 19 16:03:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:03:35.067888 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed75ae8-9cbe-48cc-bb76-c95e46b87208 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:35.199196 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed75ae8-9cbe-48cc-bb76-c95e46b87208 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:35.247126 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17a2233-72bc-4623-9242-f0a134326685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:35.248156 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17a2233-72bc-4623-9242-f0a134326685 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:35.248637 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17a2233-72bc-4623-9242-f0a134326685 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:35.249126 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17a2233-72bc-4623-9242-f0a134326685 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:35.328515 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:03:35.338048 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17a2233-72bc-4623-9242-f0a134326685 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:35.340285 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17a2233-72bc-4623-9242-f0a134326685 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:35.475753 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 619/1241] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 19 16:03:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 16:03:35.485873 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f1d2ab-00cc-4bb8-ae91-b57ac23a3593 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:35.529129 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:03:35.641591 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f1d2ab-00cc-4bb8-ae91-b57ac23a3593 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:35.687703 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58acb422-bf44-4d94-a08a-5cbb6e063ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:35.688581 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58acb422-bf44-4d94-a08a-5cbb6e063ea3 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:35.688897 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58acb422-bf44-4d94-a08a-5cbb6e063ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:35.689128 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58acb422-bf44-4d94-a08a-5cbb6e063ea3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:35.703817 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58acb422-bf44-4d94-a08a-5cbb6e063ea3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:35.705738 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58acb422-bf44-4d94-a08a-5cbb6e063ea3 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:35.723536 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-58acb422-bf44-4d94-a08a-5cbb6e063ea3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:03:35.723536 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:03:35.723536 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:03:35.723536 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:03:35.723536 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:03:35.723536 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:03:35.723536 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:03:35.723536 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:03:35.723536 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:03:35.723536 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:03:35.723536 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:03:35.723536 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:03:35.723536 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:03:35.723536 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:03:35.723536 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:03:35.723536 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:03:35.724927 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:03:35.724927 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:03:35.724927 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:03:35.724927 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:03:35.724927 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:03:35.724927 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:03:35.724927 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:03:35.724927 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:03:35.724927 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:03:35.724927 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:03:35.724927 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:03:35.724927 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:03:35.724927 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:03:35.724927 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:03:35.724927 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:03:35.724927 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:03:35.724927 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:03:35.727239 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:03:35.727239 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:03:35.727239 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:03:35.727239 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 623/1242] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:03:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:03:35.731580 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17a2233-72bc-4623-9242-f0a134326685 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:35.876557 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:03:35.884305 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17a2233-72bc-4623-9242-f0a134326685 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:35.925261 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca79915e-b527-481d-a08c-035a621f3c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:35.926108 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79915e-b527-481d-a08c-035a621f3c94 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:35.926590 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79915e-b527-481d-a08c-035a621f3c94 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:35.927009 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79915e-b527-481d-a08c-035a621f3c94 None admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:35.927923 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca79915e-b527-481d-a08c-035a621f3c94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:03:35.928335 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca79915e-b527-481d-a08c-035a621f3c94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:35.930732 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca79915e-b527-481d-a08c-035a621f3c94 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:35.950282 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 620/1243] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:03:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:03:35.957996 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58acb422-bf44-4d94-a08a-5cbb6e063ea3 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:36.066101 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:03:36.095123 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58acb422-bf44-4d94-a08a-5cbb6e063ea3 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:36.141909 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a216aeb-91a2-4b82-a1af-e44c7df7e1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:36.142623 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a216aeb-91a2-4b82-a1af-e44c7df7e1cc None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:36.142905 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a216aeb-91a2-4b82-a1af-e44c7df7e1cc None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:36.143277 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a216aeb-91a2-4b82-a1af-e44c7df7e1cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:36.152757 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a216aeb-91a2-4b82-a1af-e44c7df7e1cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:36.154350 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a216aeb-91a2-4b82-a1af-e44c7df7e1cc None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:36.168456 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-5a216aeb-91a2-4b82-a1af-e44c7df7e1cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:03:36.168456 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:03:36.168456 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:03:36.168456 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:03:36.168456 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:03:36.168456 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:03:36.168456 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:03:36.168456 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:03:36.168456 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:03:36.168456 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:03:36.168456 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:03:36.168456 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:03:36.168456 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 16:03:36.168456 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 16:03:36.168456 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 16:03:36.170401 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 16:03:36.170401 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:03:36.170401 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:03:36.170401 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:03:36.170401 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:03:36.170401 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:03:36.170401 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:03:36.170401 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:03:36.170401 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:03:36.170401 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:03:36.170401 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:03:36.170401 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:03:36.170401 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:03:36.170401 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:03:36.170401 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:03:36.170401 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:03:36.170401 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:03:36.172190 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 16:03:36.172190 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 16:03:36.172190 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 16:03:36.172190 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 16:03:36.172190 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 16:03:36.172190 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 16:03:36.172190 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:03:36.172190 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:03:36.172190 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 624/1244] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 19 16:03:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:03:36.178668 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca79915e-b527-481d-a08c-035a621f3c94 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:36.312318 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca79915e-b527-481d-a08c-035a621f3c94 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:36.359763 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a601b97-aa4c-422e-a342-69acb25ebd2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:36.360279 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a601b97-aa4c-422e-a342-69acb25ebd2d None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:36.360752 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a601b97-aa4c-422e-a342-69acb25ebd2d None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:36.360854 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a601b97-aa4c-422e-a342-69acb25ebd2d None admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:36.361518 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a601b97-aa4c-422e-a342-69acb25ebd2d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:03:36.362055 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a601b97-aa4c-422e-a342-69acb25ebd2d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:36.363291 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a601b97-aa4c-422e-a342-69acb25ebd2d None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:36.377527 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 621/1245] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 19 16:03:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:03:36.384914 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a216aeb-91a2-4b82-a1af-e44c7df7e1cc None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:36.514600 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a216aeb-91a2-4b82-a1af-e44c7df7e1cc None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:36.575784 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03afe059-1703-4277-964e-13a9c4742a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:36.576622 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03afe059-1703-4277-964e-13a9c4742a76 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:36.576997 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03afe059-1703-4277-964e-13a9c4742a76 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:36.577359 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03afe059-1703-4277-964e-13a9c4742a76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:36.578261 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03afe059-1703-4277-964e-13a9c4742a76 None admin] RBAC: Adding query filter params (role.id=1e475e6a170a4dbabc4ce8348eeb30b9, scope.project.id=be03e5300b3343169f35c643c93d1e06, user.id=1f1a985495ed48a4acc48e2e6fc7f977) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:03:36.578804 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03afe059-1703-4277-964e-13a9c4742a76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:36.580662 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03afe059-1703-4277-964e-13a9c4742a76 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:36.588448 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 625/1246] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 19 16:03:36 2022] GET /identity/v3/role_assignments?user.id=1f1a985495ed48a4acc48e2e6fc7f977&scope.project.id=be03e5300b3343169f35c643c93d1e06&role.id=1e475e6a170a4dbabc4ce8348eeb30b9 => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:03:39.169997 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad34465b-52ff-413a-9c5f-fa5af333a58f None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:39.170872 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad34465b-52ff-413a-9c5f-fa5af333a58f None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:39.170872 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad34465b-52ff-413a-9c5f-fa5af333a58f None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:39.172782 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 622/1247] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:03:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:03:39.178128 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7ffef0-b49f-4ba3-9b7a-9122e926aba9 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:39.178699 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7ffef0-b49f-4ba3-9b7a-9122e926aba9 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:39.178699 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7ffef0-b49f-4ba3-9b7a-9122e926aba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:39.188015 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:03:39.201948 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-7b7ffef0-b49f-4ba3-9b7a-9122e926aba9 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:03:39.203190 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-7b7ffef0-b49f-4ba3-9b7a-9122e926aba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:03:39.287257 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 626/1248] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:03:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 16:03:39.585789 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad34465b-52ff-413a-9c5f-fa5af333a58f None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:39.607601 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:03:39.705767 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad34465b-52ff-413a-9c5f-fa5af333a58f None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:39.742854 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2d8384-e2d4-45f3-9bbe-5185eada9bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:39.743483 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2d8384-e2d4-45f3-9bbe-5185eada9bd2 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:39.743756 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2d8384-e2d4-45f3-9bbe-5185eada9bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:39.744023 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2d8384-e2d4-45f3-9bbe-5185eada9bd2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:39.758980 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2d8384-e2d4-45f3-9bbe-5185eada9bd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 16:03:39.764316 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-bb2d8384-e2d4-45f3-9bbe-5185eada9bd2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 16:03:39.764316 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:03:39.764316 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:03:39.764316 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:03:39.764316 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:03:39.764316 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:03:39.764316 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:03:39.764316 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:03:39.764316 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:03:39.764316 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:03:39.764316 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:03:39.764316 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:03:39.764316 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 16:03:39.764316 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 16:03:39.764316 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 16:03:39.764316 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 16:03:39.765820 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 16:03:39.765820 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 16:03:39.765820 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 16:03:39.765820 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 16:03:39.765820 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:03:39.765820 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:03:39.765820 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:03:39.765820 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:03:39.765820 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:03:39.765820 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:03:39.765820 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:03:39.765820 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:03:39.765820 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:03:39.765820 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:03:39.765820 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:03:39.765820 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:03:39.766911 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:03:39.766911 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:03:39.766911 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:03:39.766911 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:03:39.766911 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 16:03:39.766911 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 16:03:39.766911 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 16:03:39.766911 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 16:03:39.766911 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 16:03:39.766911 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 16:03:39.766911 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 16:03:39.766911 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:03:39.766911 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 623/1249] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 19 16:03:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:03:39.772290 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7ffef0-b49f-4ba3-9b7a-9122e926aba9 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:39.915107 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7ffef0-b49f-4ba3-9b7a-9122e926aba9 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:39.956810 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:03:39.963630 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9996fd6a-ce3a-4d36-b239-2651a1040af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:39.964518 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9996fd6a-ce3a-4d36-b239-2651a1040af5 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:39.964993 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9996fd6a-ce3a-4d36-b239-2651a1040af5 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:39.965390 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9996fd6a-ce3a-4d36-b239-2651a1040af5 None admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:39.966347 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9996fd6a-ce3a-4d36-b239-2651a1040af5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:03:39.966765 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9996fd6a-ce3a-4d36-b239-2651a1040af5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:39.968778 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9996fd6a-ce3a-4d36-b239-2651a1040af5 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:39.979444 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 627/1250] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 16:03:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:03:39.986844 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2d8384-e2d4-45f3-9bbe-5185eada9bd2 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:40.080965 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:03:40.106817 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2d8384-e2d4-45f3-9bbe-5185eada9bd2 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:40.145608 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e94e5a-9c8a-4fb1-b613-3bd46d40d1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:40.146290 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e94e5a-9c8a-4fb1-b613-3bd46d40d1c3 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:40.146749 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e94e5a-9c8a-4fb1-b613-3bd46d40d1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:40.147028 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e94e5a-9c8a-4fb1-b613-3bd46d40d1c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:40.161489 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e94e5a-9c8a-4fb1-b613-3bd46d40d1c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:40.164426 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e94e5a-9c8a-4fb1-b613-3bd46d40d1c3 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:40.186887 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-56e94e5a-9c8a-4fb1-b613-3bd46d40d1c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:03:40.186887 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:03:40.186887 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:03:40.186887 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:03:40.186887 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:03:40.186887 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:03:40.186887 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:03:40.186887 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:03:40.186887 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:03:40.186887 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:03:40.186887 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:03:40.186887 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:03:40.186887 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:03:40.186887 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:03:40.186887 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:03:40.186887 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:03:40.188472 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:03:40.188472 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:03:40.188472 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:03:40.188472 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:03:40.188472 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:03:40.188472 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:03:40.188472 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:03:40.188472 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:03:40.188472 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:03:40.188472 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:03:40.188472 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:03:40.188472 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:03:40.188472 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:03:40.188472 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:03:40.188472 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:03:40.188472 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:03:40.188472 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:03:40.190145 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:03:40.190145 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:03:40.190145 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:03:40.190145 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 624/1251] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:03:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 16:03:40.198470 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9996fd6a-ce3a-4d36-b239-2651a1040af5 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:40.320630 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9996fd6a-ce3a-4d36-b239-2651a1040af5 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:40.375591 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5eb7ca-d56f-4d03-b8b3-f49f8d02408b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:40.376538 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5eb7ca-d56f-4d03-b8b3-f49f8d02408b None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:40.376881 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5eb7ca-d56f-4d03-b8b3-f49f8d02408b None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:40.377331 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5eb7ca-d56f-4d03-b8b3-f49f8d02408b None admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:40.378486 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5eb7ca-d56f-4d03-b8b3-f49f8d02408b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:03:40.378866 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5eb7ca-d56f-4d03-b8b3-f49f8d02408b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:40.380852 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5eb7ca-d56f-4d03-b8b3-f49f8d02408b None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:40.402439 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 628/1252] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:03:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:03:40.407700 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e94e5a-9c8a-4fb1-b613-3bd46d40d1c3 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:40.547240 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e94e5a-9c8a-4fb1-b613-3bd46d40d1c3 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:40.584627 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:03:40.596644 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204acdab-99b3-45f9-b3ed-a2359fce6160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:40.597319 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204acdab-99b3-45f9-b3ed-a2359fce6160 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:40.597598 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204acdab-99b3-45f9-b3ed-a2359fce6160 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:40.597874 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204acdab-99b3-45f9-b3ed-a2359fce6160 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:40.608747 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204acdab-99b3-45f9-b3ed-a2359fce6160 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:40.611420 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204acdab-99b3-45f9-b3ed-a2359fce6160 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:40.628518 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-204acdab-99b3-45f9-b3ed-a2359fce6160 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 19 16:03:40.628518 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:03:40.628518 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:03:40.628518 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:03:40.628518 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:03:40.628518 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:03:40.628518 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:03:40.628518 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:03:40.628518 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:03:40.628518 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:03:40.628518 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:03:40.628518 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:03:40.628518 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 16:03:40.628518 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 16:03:40.628518 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 16:03:40.628518 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 16:03:40.629685 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:03:40.629685 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:03:40.629685 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 16:03:40.629685 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:03:40.629685 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 16:03:40.629685 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:03:40.629685 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:03:40.629685 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:03:40.629685 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:03:40.629685 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:03:40.629685 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:03:40.629685 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:03:40.629685 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:03:40.629685 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:03:40.629685 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:03:40.629685 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:03:40.629685 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:03:40.629685 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:03:40.631015 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:03:40.631015 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:03:40.631015 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 16:03:40.631015 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 16:03:40.631015 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 16:03:40.631015 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 16:03:40.631015 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 16:03:40.631015 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 16:03:40.631015 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 19 16:03:40.631015 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:03:40.631015 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 625/1253] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:03:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:03:40.636573 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5eb7ca-d56f-4d03-b8b3-f49f8d02408b None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:40.700375 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:03:40.752452 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5eb7ca-d56f-4d03-b8b3-f49f8d02408b None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:40.795573 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3f0952-27f2-41dc-90a9-6fb7fab7a5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:40.796211 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f0952-27f2-41dc-90a9-6fb7fab7a5aa None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:40.796493 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f0952-27f2-41dc-90a9-6fb7fab7a5aa None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:40.796767 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f0952-27f2-41dc-90a9-6fb7fab7a5aa None admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:40.797629 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3f0952-27f2-41dc-90a9-6fb7fab7a5aa None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:03:40.797923 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3f0952-27f2-41dc-90a9-6fb7fab7a5aa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:40.799562 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3f0952-27f2-41dc-90a9-6fb7fab7a5aa None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:40.821134 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 629/1254] 10.222.0.57 () {56 vars in 1143 bytes} [Mon Sep 19 16:03:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:03:40.827047 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204acdab-99b3-45f9-b3ed-a2359fce6160 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:40.949702 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204acdab-99b3-45f9-b3ed-a2359fce6160 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:40.994216 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd84ee41-3a4a-4e56-b66c-516e2f7b9341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:40.994871 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd84ee41-3a4a-4e56-b66c-516e2f7b9341 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:40.995145 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd84ee41-3a4a-4e56-b66c-516e2f7b9341 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:40.995415 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd84ee41-3a4a-4e56-b66c-516e2f7b9341 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:41.059069 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:03:41.073947 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd84ee41-3a4a-4e56-b66c-516e2f7b9341 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:41.075444 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd84ee41-3a4a-4e56-b66c-516e2f7b9341 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:41.190845 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 626/1255] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 19 16:03:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 16:03:41.197528 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3f0952-27f2-41dc-90a9-6fb7fab7a5aa None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:41.303631 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3f0952-27f2-41dc-90a9-6fb7fab7a5aa None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:41.318984 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:03:41.347549 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da74603f-7386-4fbe-9e8e-4cd6cca68c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:41.348184 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74603f-7386-4fbe-9e8e-4cd6cca68c03 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:41.348460 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74603f-7386-4fbe-9e8e-4cd6cca68c03 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:41.348729 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74603f-7386-4fbe-9e8e-4cd6cca68c03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:41.361931 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da74603f-7386-4fbe-9e8e-4cd6cca68c03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:41.363661 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da74603f-7386-4fbe-9e8e-4cd6cca68c03 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:41.381479 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-da74603f-7386-4fbe-9e8e-4cd6cca68c03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:03:41.381479 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:03:41.381479 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:03:41.381479 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:03:41.381479 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:03:41.381479 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:03:41.381479 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:03:41.381479 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:03:41.381479 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:03:41.381479 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:03:41.381479 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:03:41.381479 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:03:41.381479 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:03:41.381479 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:03:41.381479 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:03:41.381479 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:03:41.383181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:03:41.383181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:03:41.383181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:03:41.383181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:03:41.383181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:03:41.383181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:03:41.383181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:03:41.383181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:03:41.383181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:03:41.383181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:03:41.383181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:03:41.383181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:03:41.383181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:03:41.383181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:03:41.383181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:03:41.383181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:03:41.383181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:03:41.384075 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:03:41.384075 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:03:41.384075 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:03:41.384075 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 630/1256] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:03:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:03:41.389114 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd84ee41-3a4a-4e56-b66c-516e2f7b9341 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:41.484678 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:03:41.497292 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd84ee41-3a4a-4e56-b66c-516e2f7b9341 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:41.532877 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee1ac99-ef8d-45b8-a82a-50936bb31331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:41.533488 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee1ac99-ef8d-45b8-a82a-50936bb31331 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:41.533771 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee1ac99-ef8d-45b8-a82a-50936bb31331 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:41.534041 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee1ac99-ef8d-45b8-a82a-50936bb31331 None admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:41.534725 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee1ac99-ef8d-45b8-a82a-50936bb31331 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:03:41.535026 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee1ac99-ef8d-45b8-a82a-50936bb31331 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:41.536376 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee1ac99-ef8d-45b8-a82a-50936bb31331 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:41.548569 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 627/1257] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:03:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:03:41.556041 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da74603f-7386-4fbe-9e8e-4cd6cca68c03 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:41.657579 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da74603f-7386-4fbe-9e8e-4cd6cca68c03 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:41.689927 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191ebb54-6896-4ae7-be2d-bb0aa84eb030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:41.690570 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191ebb54-6896-4ae7-be2d-bb0aa84eb030 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:41.690849 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191ebb54-6896-4ae7-be2d-bb0aa84eb030 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:41.691121 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191ebb54-6896-4ae7-be2d-bb0aa84eb030 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:41.698132 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191ebb54-6896-4ae7-be2d-bb0aa84eb030 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:41.699624 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191ebb54-6896-4ae7-be2d-bb0aa84eb030 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:41.710513 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-191ebb54-6896-4ae7-be2d-bb0aa84eb030 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:03:41.710513 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:03:41.710513 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:03:41.710513 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:03:41.710513 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:03:41.710513 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:03:41.710513 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:03:41.710513 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:03:41.710513 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:03:41.710513 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:03:41.710513 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:03:41.710513 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:03:41.710513 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 16:03:41.710513 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 16:03:41.710513 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 16:03:41.711541 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 16:03:41.711541 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:03:41.711541 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:03:41.711541 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:03:41.711541 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:03:41.711541 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:03:41.711541 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:03:41.711541 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:03:41.711541 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:03:41.711541 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:03:41.711541 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:03:41.711541 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:03:41.711541 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:03:41.711541 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:03:41.711541 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:03:41.711541 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:03:41.711541 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:03:41.712605 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 16:03:41.712605 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 16:03:41.712605 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 16:03:41.712605 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 16:03:41.712605 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 16:03:41.712605 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 16:03:41.712605 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:03:41.712605 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:03:41.712605 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 631/1258] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 19 16:03:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:03:41.717460 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee1ac99-ef8d-45b8-a82a-50936bb31331 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:41.814747 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee1ac99-ef8d-45b8-a82a-50936bb31331 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:41.847476 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29607124-9913-45cf-92f9-6d65650e9871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:41.848075 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29607124-9913-45cf-92f9-6d65650e9871 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:41.848355 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29607124-9913-45cf-92f9-6d65650e9871 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:41.848663 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29607124-9913-45cf-92f9-6d65650e9871 None admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:41.849436 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29607124-9913-45cf-92f9-6d65650e9871 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:03:41.849765 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29607124-9913-45cf-92f9-6d65650e9871 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:41.851444 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29607124-9913-45cf-92f9-6d65650e9871 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:41.863072 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 628/1259] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 19 16:03:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:03:41.869764 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191ebb54-6896-4ae7-be2d-bb0aa84eb030 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:41.895047 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:03:41.972354 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191ebb54-6896-4ae7-be2d-bb0aa84eb030 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:42.006527 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588054be-aede-497c-982c-bf4e7b7cdc31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:42.007127 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588054be-aede-497c-982c-bf4e7b7cdc31 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:42.007922 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588054be-aede-497c-982c-bf4e7b7cdc31 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:42.008191 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588054be-aede-497c-982c-bf4e7b7cdc31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:42.008909 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588054be-aede-497c-982c-bf4e7b7cdc31 None admin] RBAC: Adding query filter params (role.id=63d2d197bfd6497583e94bfb065ce29b, scope.project.id=be03e5300b3343169f35c643c93d1e06, user.id=1f1a985495ed48a4acc48e2e6fc7f977) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:03:42.009215 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588054be-aede-497c-982c-bf4e7b7cdc31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:42.010663 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588054be-aede-497c-982c-bf4e7b7cdc31 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:42.016055 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 632/1260] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 19 16:03:41 2022] GET /identity/v3/role_assignments?user.id=1f1a985495ed48a4acc48e2e6fc7f977&scope.project.id=be03e5300b3343169f35c643c93d1e06&role.id=63d2d197bfd6497583e94bfb065ce29b => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:03:44.201566 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97243c03-e1f3-42dd-adc8-25ca148cee1b None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:44.202064 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97243c03-e1f3-42dd-adc8-25ca148cee1b None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:44.202064 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97243c03-e1f3-42dd-adc8-25ca148cee1b None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:44.203344 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 629/1261] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:03:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:03:44.208509 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3bad83-01f1-4ee3-b249-7dfeb036fd09 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:44.208894 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3bad83-01f1-4ee3-b249-7dfeb036fd09 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:44.209065 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3bad83-01f1-4ee3-b249-7dfeb036fd09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:44.227386 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-cd3bad83-01f1-4ee3-b249-7dfeb036fd09 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:03:44.228578 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-cd3bad83-01f1-4ee3-b249-7dfeb036fd09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:03:44.292999 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 633/1262] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:03:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 16:03:44.562150 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97243c03-e1f3-42dd-adc8-25ca148cee1b None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:44.574686 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:03:44.674338 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97243c03-e1f3-42dd-adc8-25ca148cee1b None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:44.709303 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13f29da-c1f7-4e19-a97c-ef818840fa03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:44.709935 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13f29da-c1f7-4e19-a97c-ef818840fa03 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:44.710215 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13f29da-c1f7-4e19-a97c-ef818840fa03 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:44.711043 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13f29da-c1f7-4e19-a97c-ef818840fa03 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:44.721656 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13f29da-c1f7-4e19-a97c-ef818840fa03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 16:03:44.727472 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-b13f29da-c1f7-4e19-a97c-ef818840fa03 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 16:03:44.727472 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:03:44.727472 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:03:44.727472 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:03:44.727472 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:03:44.727472 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:03:44.727472 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:03:44.727472 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:03:44.727472 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:03:44.727472 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:03:44.727472 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:03:44.727472 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:03:44.727472 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 16:03:44.727472 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 16:03:44.727472 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 16:03:44.727472 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 16:03:44.728942 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 16:03:44.728942 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 16:03:44.728942 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 16:03:44.728942 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 16:03:44.728942 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:03:44.728942 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:03:44.728942 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:03:44.728942 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:03:44.728942 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:03:44.728942 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:03:44.728942 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:03:44.728942 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:03:44.728942 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:03:44.728942 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:03:44.728942 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:03:44.728942 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:03:44.730024 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:03:44.730024 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:03:44.730024 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:03:44.730024 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:03:44.730024 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 16:03:44.730024 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 16:03:44.730024 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 16:03:44.730024 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 16:03:44.730024 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 16:03:44.730024 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 16:03:44.730024 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 16:03:44.730024 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:03:44.730024 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 630/1263] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 19 16:03:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:03:44.736136 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3bad83-01f1-4ee3-b249-7dfeb036fd09 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:44.754992 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:03:44.851668 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3bad83-01f1-4ee3-b249-7dfeb036fd09 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:44.886025 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dec600-b3fb-41fe-8295-e0206293ba34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:44.886648 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dec600-b3fb-41fe-8295-e0206293ba34 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:44.886931 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dec600-b3fb-41fe-8295-e0206293ba34 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:44.887198 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dec600-b3fb-41fe-8295-e0206293ba34 None admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:44.887857 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dec600-b3fb-41fe-8295-e0206293ba34 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:03:44.888163 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dec600-b3fb-41fe-8295-e0206293ba34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:44.889525 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dec600-b3fb-41fe-8295-e0206293ba34 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:44.898402 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 634/1264] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 16:03:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:03:44.904356 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13f29da-c1f7-4e19-a97c-ef818840fa03 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:44.988719 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:03:45.028697 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13f29da-c1f7-4e19-a97c-ef818840fa03 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:45.068718 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd700fa-f6b3-4160-8e3f-29b68310c479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:45.069425 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd700fa-f6b3-4160-8e3f-29b68310c479 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:45.070251 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd700fa-f6b3-4160-8e3f-29b68310c479 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:45.070550 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd700fa-f6b3-4160-8e3f-29b68310c479 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:45.083636 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd700fa-f6b3-4160-8e3f-29b68310c479 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:45.085112 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd700fa-f6b3-4160-8e3f-29b68310c479 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:45.108080 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-efd700fa-f6b3-4160-8e3f-29b68310c479 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:03:45.108080 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:03:45.108080 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:03:45.108080 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:03:45.108080 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:03:45.108080 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:03:45.108080 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:03:45.108080 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:03:45.108080 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:03:45.108080 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:03:45.108080 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:03:45.108080 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:03:45.108080 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:03:45.108080 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:03:45.108080 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:03:45.108080 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:03:45.109299 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:03:45.109299 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:03:45.109299 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:03:45.109299 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:03:45.109299 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:03:45.109299 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:03:45.109299 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:03:45.109299 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:03:45.109299 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:03:45.109299 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:03:45.109299 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:03:45.109299 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:03:45.109299 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:03:45.109299 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:03:45.109299 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:03:45.109299 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:03:45.109299 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:03:45.110318 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:03:45.110318 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:03:45.110318 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:03:45.110318 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 631/1265] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:03:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:03:45.116869 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dec600-b3fb-41fe-8295-e0206293ba34 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:45.204550 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:03:45.225016 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dec600-b3fb-41fe-8295-e0206293ba34 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:45.263265 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf506ff-d111-47d0-9b4a-5b744e48f4e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:45.264055 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf506ff-d111-47d0-9b4a-5b744e48f4e0 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:45.264454 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf506ff-d111-47d0-9b4a-5b744e48f4e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:45.264772 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf506ff-d111-47d0-9b4a-5b744e48f4e0 None admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:45.265493 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf506ff-d111-47d0-9b4a-5b744e48f4e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:03:45.265833 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf506ff-d111-47d0-9b4a-5b744e48f4e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:45.267925 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf506ff-d111-47d0-9b4a-5b744e48f4e0 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:45.283418 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 635/1266] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:03:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:03:45.289570 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd700fa-f6b3-4160-8e3f-29b68310c479 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:45.407029 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd700fa-f6b3-4160-8e3f-29b68310c479 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:45.431984 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:03:45.452479 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f293588-c346-4e5a-bb9c-a8bbcda0c449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:45.453127 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f293588-c346-4e5a-bb9c-a8bbcda0c449 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:45.453406 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f293588-c346-4e5a-bb9c-a8bbcda0c449 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:45.453682 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f293588-c346-4e5a-bb9c-a8bbcda0c449 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:45.463052 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f293588-c346-4e5a-bb9c-a8bbcda0c449 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:45.464497 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f293588-c346-4e5a-bb9c-a8bbcda0c449 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:45.476202 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-7f293588-c346-4e5a-bb9c-a8bbcda0c449 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 19 16:03:45.476202 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:03:45.476202 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:03:45.476202 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:03:45.476202 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:03:45.476202 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:03:45.476202 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:03:45.476202 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:03:45.476202 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:03:45.476202 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:03:45.476202 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:03:45.476202 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:03:45.476202 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 16:03:45.476202 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 16:03:45.476202 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 16:03:45.476202 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 16:03:45.477454 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:03:45.477454 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:03:45.477454 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 16:03:45.477454 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:03:45.477454 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 16:03:45.477454 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:03:45.477454 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:03:45.477454 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:03:45.477454 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:03:45.477454 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:03:45.477454 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:03:45.477454 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:03:45.477454 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:03:45.477454 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:03:45.477454 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:03:45.477454 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:03:45.477454 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:03:45.477454 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:03:45.478647 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:03:45.478647 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:03:45.478647 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 16:03:45.478647 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 16:03:45.478647 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 16:03:45.478647 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 16:03:45.478647 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 16:03:45.478647 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 16:03:45.478647 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 19 16:03:45.478647 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:03:45.478647 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 632/1267] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:03:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:03:45.483405 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf506ff-d111-47d0-9b4a-5b744e48f4e0 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:45.584043 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf506ff-d111-47d0-9b4a-5b744e48f4e0 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:45.620983 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620c923a-44c8-4bec-8d94-0b987add6fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:45.621589 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c923a-44c8-4bec-8d94-0b987add6fa7 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:45.621868 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c923a-44c8-4bec-8d94-0b987add6fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:45.622137 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c923a-44c8-4bec-8d94-0b987add6fa7 None admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:45.622904 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620c923a-44c8-4bec-8d94-0b987add6fa7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:03:45.623210 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620c923a-44c8-4bec-8d94-0b987add6fa7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:45.624622 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620c923a-44c8-4bec-8d94-0b987add6fa7 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:45.634181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:03:45.642531 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 636/1268] 10.222.0.57 () {56 vars in 1143 bytes} [Mon Sep 19 16:03:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:03:45.649779 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f293588-c346-4e5a-bb9c-a8bbcda0c449 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:45.758505 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f293588-c346-4e5a-bb9c-a8bbcda0c449 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:45.799521 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2dc761-68aa-402d-bcee-fe0ae776f443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:45.800152 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2dc761-68aa-402d-bcee-fe0ae776f443 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:45.800433 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2dc761-68aa-402d-bcee-fe0ae776f443 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:45.800707 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2dc761-68aa-402d-bcee-fe0ae776f443 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:45.814055 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2dc761-68aa-402d-bcee-fe0ae776f443 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:45.815778 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2dc761-68aa-402d-bcee-fe0ae776f443 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:45.831577 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-5d2dc761-68aa-402d-bcee-fe0ae776f443 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:03:45.831577 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:03:45.831577 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:03:45.831577 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:03:45.831577 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:03:45.831577 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:03:45.831577 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:03:45.831577 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:03:45.831577 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:03:45.831577 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:03:45.831577 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:03:45.831577 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:03:45.831577 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:03:45.831577 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:03:45.831577 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:03:45.831577 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:03:45.832725 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:03:45.832725 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:03:45.832725 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:03:45.832725 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:03:45.832725 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:03:45.832725 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:03:45.832725 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:03:45.832725 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:03:45.832725 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:03:45.832725 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:03:45.832725 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:03:45.832725 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:03:45.832725 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:03:45.832725 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:03:45.832725 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:03:45.832725 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:03:45.832725 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:03:45.833720 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:03:45.833720 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:03:45.833720 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:03:45.833720 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 633/1269] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:03:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:03:45.839152 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620c923a-44c8-4bec-8d94-0b987add6fa7 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:45.953061 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620c923a-44c8-4bec-8d94-0b987add6fa7 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:45.992317 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c257cb-6618-4285-90e1-255d68b79d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:45.993028 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c257cb-6618-4285-90e1-255d68b79d45 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:45.993362 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c257cb-6618-4285-90e1-255d68b79d45 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:45.993660 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c257cb-6618-4285-90e1-255d68b79d45 None admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:45.994335 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c257cb-6618-4285-90e1-255d68b79d45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:03:45.994744 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c257cb-6618-4285-90e1-255d68b79d45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:45.996385 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c257cb-6618-4285-90e1-255d68b79d45 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:46.009261 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 637/1270] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:03:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:03:46.015184 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2dc761-68aa-402d-bcee-fe0ae776f443 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:46.049796 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:03:46.124706 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2dc761-68aa-402d-bcee-fe0ae776f443 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:46.159655 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3faa71-3897-4324-ab46-c6a0b2760e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:46.160300 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3faa71-3897-4324-ab46-c6a0b2760e60 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:46.160599 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3faa71-3897-4324-ab46-c6a0b2760e60 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:46.160874 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3faa71-3897-4324-ab46-c6a0b2760e60 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:46.168823 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3faa71-3897-4324-ab46-c6a0b2760e60 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:46.170357 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3faa71-3897-4324-ab46-c6a0b2760e60 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:46.183651 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-ab3faa71-3897-4324-ab46-c6a0b2760e60 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:03:46.183651 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:03:46.183651 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:03:46.183651 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:03:46.183651 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:03:46.183651 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:03:46.183651 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:03:46.183651 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:03:46.183651 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:03:46.183651 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:03:46.183651 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:03:46.183651 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:03:46.183651 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 16:03:46.183651 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 16:03:46.183651 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 16:03:46.184795 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 16:03:46.184795 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:03:46.184795 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:03:46.184795 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:03:46.184795 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:03:46.184795 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:03:46.184795 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:03:46.184795 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:03:46.184795 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:03:46.184795 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:03:46.184795 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:03:46.184795 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:03:46.184795 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:03:46.184795 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:03:46.184795 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:03:46.184795 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:03:46.184795 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:03:46.186066 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 16:03:46.186066 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 16:03:46.186066 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 16:03:46.186066 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 16:03:46.186066 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 16:03:46.186066 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 16:03:46.186066 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:03:46.186066 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:03:46.186066 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 634/1271] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 19 16:03:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:03:46.191092 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c257cb-6618-4285-90e1-255d68b79d45 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:46.255997 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:03:46.302659 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c257cb-6618-4285-90e1-255d68b79d45 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:46.342816 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43a7022-afaf-4d7f-8a50-7c258884d152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:46.343497 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43a7022-afaf-4d7f-8a50-7c258884d152 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:46.343819 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43a7022-afaf-4d7f-8a50-7c258884d152 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:46.344096 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43a7022-afaf-4d7f-8a50-7c258884d152 None admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:46.344951 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43a7022-afaf-4d7f-8a50-7c258884d152 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:03:46.345215 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43a7022-afaf-4d7f-8a50-7c258884d152 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:46.346985 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43a7022-afaf-4d7f-8a50-7c258884d152 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:46.362898 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 638/1272] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 19 16:03:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:03:46.370165 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3faa71-3897-4324-ab46-c6a0b2760e60 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:46.471677 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:03:46.480289 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3faa71-3897-4324-ab46-c6a0b2760e60 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:46.514507 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f6c672-eabd-4c65-8458-4430031e1ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:46.515187 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f6c672-eabd-4c65-8458-4430031e1ca8 None admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:46.515461 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f6c672-eabd-4c65-8458-4430031e1ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:46.515730 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f6c672-eabd-4c65-8458-4430031e1ca8 None admin] PATH_INFO: `/v3/projects/be03e5300b3343169f35c643c93d1e06/users/1f1a985495ed48a4acc48e2e6fc7f977/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:46.517719 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f6c672-eabd-4c65-8458-4430031e1ca8 None admin] RBAC: Authorizing `identity:create_grant(project_id=be03e5300b3343169f35c643c93d1e06, user_id=1f1a985495ed48a4acc48e2e6fc7f977, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:46.519198 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f6c672-eabd-4c65-8458-4430031e1ca8 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:46.533415 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 635/1273] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Sep 19 16:03:46 2022] PUT /identity/v3/projects/be03e5300b3343169f35c643c93d1e06/users/1f1a985495ed48a4acc48e2e6fc7f977/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:03:48.747135 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a77c21-9af6-4ca2-a0ae-465f1665f795 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:48.747720 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a77c21-9af6-4ca2-a0ae-465f1665f795 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:48.747720 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a77c21-9af6-4ca2-a0ae-465f1665f795 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:48.749043 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 639/1274] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:03:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:03:48.755177 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da36d8b7-ce2a-416f-af2a-2c7e9b4ce605 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:48.755692 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da36d8b7-ce2a-416f-af2a-2c7e9b4ce605 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:48.755813 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da36d8b7-ce2a-416f-af2a-2c7e9b4ce605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:48.773620 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-da36d8b7-ce2a-416f-af2a-2c7e9b4ce605 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:03:48.774854 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-da36d8b7-ce2a-416f-af2a-2c7e9b4ce605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:03:48.842172 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 636/1275] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:03:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 16:03:49.116083 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a77c21-9af6-4ca2-a0ae-465f1665f795 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:49.214678 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a77c21-9af6-4ca2-a0ae-465f1665f795 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:49.231118 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:03:49.247937 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909371dc-0c24-436b-8397-c59846ee3946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:49.248555 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909371dc-0c24-436b-8397-c59846ee3946 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:49.248831 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909371dc-0c24-436b-8397-c59846ee3946 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:49.249100 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909371dc-0c24-436b-8397-c59846ee3946 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:49.259528 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909371dc-0c24-436b-8397-c59846ee3946 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 16:03:49.264181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-909371dc-0c24-436b-8397-c59846ee3946 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 16:03:49.264181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:03:49.264181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:03:49.264181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:03:49.264181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:03:49.264181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:03:49.264181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:03:49.264181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:03:49.264181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:03:49.264181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:03:49.264181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:03:49.264181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:03:49.264181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 16:03:49.264181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 16:03:49.264181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 16:03:49.264181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 16:03:49.265313 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 16:03:49.265313 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 16:03:49.265313 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 16:03:49.265313 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 16:03:49.265313 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:03:49.265313 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:03:49.265313 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:03:49.265313 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:03:49.265313 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:03:49.265313 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:03:49.265313 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:03:49.265313 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:03:49.265313 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:03:49.265313 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:03:49.265313 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:03:49.265313 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:03:49.266208 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:03:49.266208 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:03:49.266208 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:03:49.266208 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:03:49.266208 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 16:03:49.266208 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 16:03:49.266208 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 16:03:49.266208 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 16:03:49.266208 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 16:03:49.266208 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 16:03:49.266208 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 16:03:49.266208 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:03:49.266208 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 640/1276] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 19 16:03:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:03:49.271376 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da36d8b7-ce2a-416f-af2a-2c7e9b4ce605 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:49.358783 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:03:49.380262 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da36d8b7-ce2a-416f-af2a-2c7e9b4ce605 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:49.417012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834c91f7-d9fb-4fc6-87e2-e8ef2fc0f748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:49.417617 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834c91f7-d9fb-4fc6-87e2-e8ef2fc0f748 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:49.417897 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834c91f7-d9fb-4fc6-87e2-e8ef2fc0f748 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:49.418172 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834c91f7-d9fb-4fc6-87e2-e8ef2fc0f748 None admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:49.419110 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834c91f7-d9fb-4fc6-87e2-e8ef2fc0f748 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:03:49.419416 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834c91f7-d9fb-4fc6-87e2-e8ef2fc0f748 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:49.420772 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834c91f7-d9fb-4fc6-87e2-e8ef2fc0f748 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:49.428821 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 637/1277] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 19 16:03:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:03:49.435398 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909371dc-0c24-436b-8397-c59846ee3946 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:49.550767 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909371dc-0c24-436b-8397-c59846ee3946 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:49.584379 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110d2a19-3a8d-4d26-99ec-1293047f076a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:49.585000 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d2a19-3a8d-4d26-99ec-1293047f076a None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:49.585304 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d2a19-3a8d-4d26-99ec-1293047f076a None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:49.585574 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d2a19-3a8d-4d26-99ec-1293047f076a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:49.595384 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110d2a19-3a8d-4d26-99ec-1293047f076a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:49.596816 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110d2a19-3a8d-4d26-99ec-1293047f076a None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:49.611634 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-110d2a19-3a8d-4d26-99ec-1293047f076a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:03:49.611634 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:03:49.611634 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:03:49.611634 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:03:49.611634 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:03:49.611634 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:03:49.611634 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:03:49.611634 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:03:49.611634 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:03:49.611634 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:03:49.611634 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:03:49.611634 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:03:49.611634 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:03:49.611634 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:03:49.611634 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:03:49.611634 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:03:49.612805 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:03:49.612805 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:03:49.612805 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:03:49.612805 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:03:49.612805 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:03:49.612805 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:03:49.612805 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:03:49.612805 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:03:49.612805 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:03:49.612805 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:03:49.612805 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:03:49.612805 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:03:49.612805 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:03:49.612805 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:03:49.612805 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:03:49.612805 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:03:49.612805 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:03:49.613688 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:03:49.613688 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:03:49.613688 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:03:49.613688 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 641/1278] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:03:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:03:49.619192 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834c91f7-d9fb-4fc6-87e2-e8ef2fc0f748 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:49.715208 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834c91f7-d9fb-4fc6-87e2-e8ef2fc0f748 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:49.748378 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faee422b-2269-4192-b0ac-06e84a050e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:49.748969 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee422b-2269-4192-b0ac-06e84a050e17 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:49.749243 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee422b-2269-4192-b0ac-06e84a050e17 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:49.749513 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee422b-2269-4192-b0ac-06e84a050e17 None admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:49.750138 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faee422b-2269-4192-b0ac-06e84a050e17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:03:49.750502 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faee422b-2269-4192-b0ac-06e84a050e17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:49.751841 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faee422b-2269-4192-b0ac-06e84a050e17 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:49.763393 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 638/1279] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:03:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:03:49.769134 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110d2a19-3a8d-4d26-99ec-1293047f076a None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:49.799915 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:03:49.869162 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110d2a19-3a8d-4d26-99ec-1293047f076a None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:49.913860 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2d507c-754d-4a70-ba13-1ffb55850793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:49.915025 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2d507c-754d-4a70-ba13-1ffb55850793 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:49.915302 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2d507c-754d-4a70-ba13-1ffb55850793 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:49.915571 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2d507c-754d-4a70-ba13-1ffb55850793 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:49.923735 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2d507c-754d-4a70-ba13-1ffb55850793 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:49.925161 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2d507c-754d-4a70-ba13-1ffb55850793 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:49.936278 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-fa2d507c-754d-4a70-ba13-1ffb55850793 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 19 16:03:49.936278 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:03:49.936278 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:03:49.936278 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:03:49.936278 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:03:49.936278 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:03:49.936278 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:03:49.936278 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:03:49.936278 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:03:49.936278 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:03:49.936278 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:03:49.936278 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:03:49.936278 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 16:03:49.936278 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 16:03:49.936278 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 16:03:49.936278 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 16:03:49.937633 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:03:49.937633 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:03:49.937633 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 16:03:49.937633 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:03:49.937633 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 16:03:49.937633 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 16:03:49.937633 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:03:49.937633 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:03:49.937633 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:03:49.937633 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:03:49.937633 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:03:49.937633 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:03:49.937633 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:03:49.937633 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:03:49.937633 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:03:49.937633 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:03:49.937633 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:03:49.937633 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:03:49.938715 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:03:49.938715 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:03:49.938715 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 16:03:49.938715 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 16:03:49.938715 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 16:03:49.938715 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 16:03:49.938715 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 16:03:49.938715 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 16:03:49.938715 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 19 16:03:49.938715 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:03:49.938715 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 642/1280] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:03:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:03:49.945606 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faee422b-2269-4192-b0ac-06e84a050e17 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:49.950701 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:03:50.049969 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faee422b-2269-4192-b0ac-06e84a050e17 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:50.082136 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5b3539-07c7-40e5-97a5-485b266ba167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:50.082756 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b3539-07c7-40e5-97a5-485b266ba167 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:50.083034 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b3539-07c7-40e5-97a5-485b266ba167 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:50.083305 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b3539-07c7-40e5-97a5-485b266ba167 None admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:50.084015 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5b3539-07c7-40e5-97a5-485b266ba167 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:03:50.084320 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5b3539-07c7-40e5-97a5-485b266ba167 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:50.086250 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5b3539-07c7-40e5-97a5-485b266ba167 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:50.103906 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 639/1281] 10.222.0.57 () {56 vars in 1143 bytes} [Mon Sep 19 16:03:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:03:50.109720 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2d507c-754d-4a70-ba13-1ffb55850793 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:50.200703 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:03:50.211423 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2d507c-754d-4a70-ba13-1ffb55850793 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:50.247304 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d603da-1aa3-45cf-bf46-a3e43dec5a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:50.247928 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d603da-1aa3-45cf-bf46-a3e43dec5a7a None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:50.248207 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d603da-1aa3-45cf-bf46-a3e43dec5a7a None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:50.248482 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d603da-1aa3-45cf-bf46-a3e43dec5a7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:50.319945 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d603da-1aa3-45cf-bf46-a3e43dec5a7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:50.324488 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d603da-1aa3-45cf-bf46-a3e43dec5a7a None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:50.411629 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:03:50.423094 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 643/1282] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 19 16:03:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 16:03:50.428929 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5b3539-07c7-40e5-97a5-485b266ba167 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:50.487759 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:03:50.525319 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5b3539-07c7-40e5-97a5-485b266ba167 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:50.559647 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b415ff12-74fa-4444-be8b-d33a75039b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:50.560296 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b415ff12-74fa-4444-be8b-d33a75039b9e None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:50.560576 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b415ff12-74fa-4444-be8b-d33a75039b9e None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:50.560848 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b415ff12-74fa-4444-be8b-d33a75039b9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:50.573295 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b415ff12-74fa-4444-be8b-d33a75039b9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:50.574717 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b415ff12-74fa-4444-be8b-d33a75039b9e None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:50.591625 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-b415ff12-74fa-4444-be8b-d33a75039b9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:03:50.591625 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:03:50.591625 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:03:50.591625 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:03:50.591625 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:03:50.591625 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:03:50.591625 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:03:50.591625 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:03:50.591625 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:03:50.591625 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:03:50.591625 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:03:50.591625 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:03:50.591625 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 16:03:50.591625 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 16:03:50.591625 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 16:03:50.591625 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 16:03:50.592904 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:03:50.592904 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:03:50.592904 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:03:50.592904 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:03:50.592904 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:03:50.592904 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:03:50.592904 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:03:50.592904 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:03:50.592904 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:03:50.592904 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:03:50.592904 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:03:50.592904 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:03:50.592904 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:03:50.592904 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:03:50.592904 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:03:50.592904 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:03:50.592904 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 16:03:50.594144 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 16:03:50.594144 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 16:03:50.594144 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:03:50.594144 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 640/1283] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:03:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:03:50.598841 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d603da-1aa3-45cf-bf46-a3e43dec5a7a None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:50.701866 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d603da-1aa3-45cf-bf46-a3e43dec5a7a None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:50.738752 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb53ef0-16ec-48e5-8f91-b0b3fe0831d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:50.739416 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb53ef0-16ec-48e5-8f91-b0b3fe0831d0 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:50.739713 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb53ef0-16ec-48e5-8f91-b0b3fe0831d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:50.739984 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb53ef0-16ec-48e5-8f91-b0b3fe0831d0 None admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:50.740633 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb53ef0-16ec-48e5-8f91-b0b3fe0831d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:03:50.740937 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb53ef0-16ec-48e5-8f91-b0b3fe0831d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:50.742435 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb53ef0-16ec-48e5-8f91-b0b3fe0831d0 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:50.754752 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 644/1284] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 19 16:03:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:03:50.760567 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b415ff12-74fa-4444-be8b-d33a75039b9e None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:50.864866 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b415ff12-74fa-4444-be8b-d33a75039b9e None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:50.879485 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:03:50.901042 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ace9495-9ea5-4d10-894f-ebc564aff7a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:50.901688 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace9495-9ea5-4d10-894f-ebc564aff7a6 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:50.901962 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace9495-9ea5-4d10-894f-ebc564aff7a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:50.902230 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace9495-9ea5-4d10-894f-ebc564aff7a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:50.909819 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ace9495-9ea5-4d10-894f-ebc564aff7a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:50.911302 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ace9495-9ea5-4d10-894f-ebc564aff7a6 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:50.924357 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-1ace9495-9ea5-4d10-894f-ebc564aff7a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:03:50.924357 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:03:50.924357 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:03:50.924357 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:03:50.924357 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:03:50.924357 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:03:50.924357 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:03:50.924357 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:03:50.924357 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:03:50.924357 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:03:50.924357 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:03:50.924357 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:03:50.924357 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 16:03:50.924357 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 16:03:50.924357 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 16:03:50.925511 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 16:03:50.925511 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:03:50.925511 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:03:50.925511 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:03:50.925511 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:03:50.925511 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:03:50.925511 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:03:50.925511 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:03:50.925511 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:03:50.925511 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:03:50.925511 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:03:50.925511 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:03:50.925511 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:03:50.925511 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:03:50.925511 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:03:50.925511 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:03:50.925511 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:03:50.926472 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 16:03:50.926472 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 16:03:50.926472 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 16:03:50.926472 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 16:03:50.926472 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 16:03:50.926472 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 16:03:50.926472 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 16:03:50.926472 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:03:50.926472 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 641/1285] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 19 16:03:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:03:50.931669 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb53ef0-16ec-48e5-8f91-b0b3fe0831d0 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:51.005573 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:03:51.067051 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb53ef0-16ec-48e5-8f91-b0b3fe0831d0 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:51.101861 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba3886f-9aa3-4d31-a4fb-f78f0495e0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:51.102495 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba3886f-9aa3-4d31-a4fb-f78f0495e0ec None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:51.102776 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba3886f-9aa3-4d31-a4fb-f78f0495e0ec None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:51.103050 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba3886f-9aa3-4d31-a4fb-f78f0495e0ec None admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:51.103743 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba3886f-9aa3-4d31-a4fb-f78f0495e0ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:03:51.104048 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba3886f-9aa3-4d31-a4fb-f78f0495e0ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:51.105475 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba3886f-9aa3-4d31-a4fb-f78f0495e0ec None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:51.118127 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 645/1286] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 19 16:03:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:03:51.124272 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ace9495-9ea5-4d10-894f-ebc564aff7a6 None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:51.223270 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ace9495-9ea5-4d10-894f-ebc564aff7a6 None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:51.264083 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30056813-1d88-4d82-9532-c95601bab2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:51.264709 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30056813-1d88-4d82-9532-c95601bab2ab None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:51.264986 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30056813-1d88-4d82-9532-c95601bab2ab None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:51.265259 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30056813-1d88-4d82-9532-c95601bab2ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:51.266003 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30056813-1d88-4d82-9532-c95601bab2ab None admin] RBAC: Adding query filter params (role.id=63d2d197bfd6497583e94bfb065ce29b, scope.project.id=be03e5300b3343169f35c643c93d1e06, user.id=1f1a985495ed48a4acc48e2e6fc7f977) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:03:51.266308 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30056813-1d88-4d82-9532-c95601bab2ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:51.268171 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30056813-1d88-4d82-9532-c95601bab2ab None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:51.276394 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 642/1287] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 19 16:03:51 2022] GET /identity/v3/role_assignments?user.id=1f1a985495ed48a4acc48e2e6fc7f977&scope.project.id=be03e5300b3343169f35c643c93d1e06&role.id=63d2d197bfd6497583e94bfb065ce29b => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:03:53.469329 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27101579-bd28-4e3f-ba49-4f266167d255 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:53.469881 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27101579-bd28-4e3f-ba49-4f266167d255 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:53.469881 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27101579-bd28-4e3f-ba49-4f266167d255 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:53.471238 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 646/1288] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:03:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:03:53.476314 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb504b8-edd4-40b5-870d-1e1b8909e428 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:53.476716 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb504b8-edd4-40b5-870d-1e1b8909e428 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:53.476857 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb504b8-edd4-40b5-870d-1e1b8909e428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:53.494603 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-4eb504b8-edd4-40b5-870d-1e1b8909e428 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:03:53.495758 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-4eb504b8-edd4-40b5-870d-1e1b8909e428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:03:53.524774 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:03:53.561433 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 643/1289] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:03:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 16:03:53.825774 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27101579-bd28-4e3f-ba49-4f266167d255 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:53.924348 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27101579-bd28-4e3f-ba49-4f266167d255 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:53.937422 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:03:53.969097 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa468d3-4298-47e0-9ded-588a42b3f9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:53.969936 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa468d3-4298-47e0-9ded-588a42b3f9d7 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:53.970217 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa468d3-4298-47e0-9ded-588a42b3f9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:53.970528 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa468d3-4298-47e0-9ded-588a42b3f9d7 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:53.971216 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa468d3-4298-47e0-9ded-588a42b3f9d7 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:53.973092 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa468d3-4298-47e0-9ded-588a42b3f9d7 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:53.986503 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-2aa468d3-4298-47e0-9ded-588a42b3f9d7 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 19 16:03:53.986503 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:03:53.986503 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 16:03:53.986503 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 16:03:53.986503 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 16:03:53.986503 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 16:03:53.986503 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 16:03:53.986503 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 16:03:53.986503 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 19 16:03:53.986503 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:03:53.986503 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 16:03:53.986503 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:03:53.986503 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:03:53.986503 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:03:53.986503 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:03:53.986503 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:03:53.986503 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:03:53.987944 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:03:53.987944 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:03:53.987944 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:03:53.987944 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:03:53.987944 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:03:53.987944 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:03:53.987944 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 16:03:53.987944 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 16:03:53.987944 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 16:03:53.987944 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 16:03:53.987944 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:03:53.987944 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:03:53.987944 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:03:53.987944 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:03:53.987944 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:03:53.987944 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:03:53.989148 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:03:53.989148 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:03:53.989148 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:03:53.989148 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:03:53.989148 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:03:53.989148 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:03:53.989148 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:03:53.989148 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:03:53.989148 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:03:53.989148 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:03:53.989148 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 16:03:53.989148 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 16:03:53.989148 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 19 16:03:53.989148 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:03:53.989148 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 647/1290] 10.222.0.57 () {56 vars in 1117 bytes} [Mon Sep 19 16:03:53 2022] GET /identity/v3/services/placement => generated 90 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:03:53.994629 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb504b8-edd4-40b5-870d-1e1b8909e428 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:54.095017 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb504b8-edd4-40b5-870d-1e1b8909e428 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:54.128759 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2148d19-9133-4fa7-bff9-8a3166ebdbd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:54.129363 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2148d19-9133-4fa7-bff9-8a3166ebdbd4 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:54.129635 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2148d19-9133-4fa7-bff9-8a3166ebdbd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:54.129903 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2148d19-9133-4fa7-bff9-8a3166ebdbd4 None admin] PATH_INFO: `/v3/services` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:54.130551 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2148d19-9133-4fa7-bff9-8a3166ebdbd4 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:03:54.130852 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2148d19-9133-4fa7-bff9-8a3166ebdbd4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:54.132238 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2148d19-9133-4fa7-bff9-8a3166ebdbd4 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:54.136949 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 644/1291] 10.222.0.57 () {56 vars in 1116 bytes} [Mon Sep 19 16:03:53 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:03:54.142860 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa468d3-4298-47e0-9ded-588a42b3f9d7 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:54.238892 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa468d3-4298-47e0-9ded-588a42b3f9d7 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:54.271784 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f535bef7-f0da-4008-a57b-25db40cc4e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:54.272386 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f535bef7-f0da-4008-a57b-25db40cc4e3b None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:54.272669 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f535bef7-f0da-4008-a57b-25db40cc4e3b None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:54.272942 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f535bef7-f0da-4008-a57b-25db40cc4e3b None admin] PATH_INFO: `/v3/services` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:54.273571 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f535bef7-f0da-4008-a57b-25db40cc4e3b None admin] RBAC: Adding query filter params (type=placement) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:03:54.273877 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f535bef7-f0da-4008-a57b-25db40cc4e3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:54.275276 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f535bef7-f0da-4008-a57b-25db40cc4e3b None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:54.279850 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 648/1292] 10.222.0.57 () {56 vars in 1116 bytes} [Mon Sep 19 16:03:54 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:03:56.523039 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d2e306-c4db-4a9f-9f0f-37eac6a3f77c None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:56.523593 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d2e306-c4db-4a9f-9f0f-37eac6a3f77c None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:56.523593 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d2e306-c4db-4a9f-9f0f-37eac6a3f77c None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:56.524919 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 645/1293] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:03:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:03:56.530079 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74340006-7f95-43de-a21d-f47cc1c522e9 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:56.530494 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74340006-7f95-43de-a21d-f47cc1c522e9 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:56.530656 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74340006-7f95-43de-a21d-f47cc1c522e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:56.549263 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-74340006-7f95-43de-a21d-f47cc1c522e9 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:03:56.550490 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-74340006-7f95-43de-a21d-f47cc1c522e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:03:56.579665 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:03:56.621026 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 649/1294] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:03:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 16:03:56.888138 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d2e306-c4db-4a9f-9f0f-37eac6a3f77c None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:56.919841 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:03:56.994019 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d2e306-c4db-4a9f-9f0f-37eac6a3f77c None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:57.032673 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60da819d-2771-431e-aa40-d0f121b2d823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:57.033198 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60da819d-2771-431e-aa40-d0f121b2d823 None admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:57.033475 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60da819d-2771-431e-aa40-d0f121b2d823 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:57.033742 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60da819d-2771-431e-aa40-d0f121b2d823 None admin] PATH_INFO: `/v3/services` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:57.034414 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60da819d-2771-431e-aa40-d0f121b2d823 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:57.035763 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60da819d-2771-431e-aa40-d0f121b2d823 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:57.050499 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 646/1295] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Sep 19 16:03:56 2022] POST /identity/v3/services => generated 246 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:03:59.260469 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa19bd56-6353-4fc8-b9ed-f20f98d2d16b None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:59.261007 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa19bd56-6353-4fc8-b9ed-f20f98d2d16b None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:59.261007 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa19bd56-6353-4fc8-b9ed-f20f98d2d16b None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:59.262342 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 650/1296] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:03:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:03:59.267833 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c6d576-255b-4835-b1fe-4d240827600e None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:59.268327 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c6d576-255b-4835-b1fe-4d240827600e None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:59.268424 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c6d576-255b-4835-b1fe-4d240827600e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:59.288726 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-98c6d576-255b-4835-b1fe-4d240827600e None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:03:59.290172 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-98c6d576-255b-4835-b1fe-4d240827600e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:03:59.367971 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 647/1297] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:03:59 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 16:03:59.635847 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa19bd56-6353-4fc8-b9ed-f20f98d2d16b None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:59.741432 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa19bd56-6353-4fc8-b9ed-f20f98d2d16b None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:59.777431 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eefa00a-1154-4ea2-84a3-1b141d6f7e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:59.778059 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eefa00a-1154-4ea2-84a3-1b141d6f7e42 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:59.778336 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eefa00a-1154-4ea2-84a3-1b141d6f7e42 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:59.778637 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eefa00a-1154-4ea2-84a3-1b141d6f7e42 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:59.779215 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eefa00a-1154-4ea2-84a3-1b141d6f7e42 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:59.780583 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eefa00a-1154-4ea2-84a3-1b141d6f7e42 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:59.790086 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-8eefa00a-1154-4ea2-84a3-1b141d6f7e42 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 19 16:03:59.790086 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:03:59.790086 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 16:03:59.790086 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 16:03:59.790086 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 16:03:59.790086 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 16:03:59.790086 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 16:03:59.790086 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 16:03:59.790086 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 19 16:03:59.790086 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:03:59.790086 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 16:03:59.790086 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:03:59.790086 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:03:59.790086 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:03:59.790086 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:03:59.790086 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:03:59.790086 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:03:59.792105 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:03:59.792105 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:03:59.792105 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:03:59.792105 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:03:59.792105 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:03:59.792105 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:03:59.792105 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 16:03:59.792105 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 16:03:59.792105 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 16:03:59.792105 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 16:03:59.792105 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:03:59.792105 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:03:59.792105 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:03:59.792105 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:03:59.792105 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:03:59.792105 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:03:59.793165 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:03:59.793165 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:03:59.793165 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:03:59.793165 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:03:59.793165 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:03:59.793165 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:03:59.793165 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:03:59.793165 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:03:59.793165 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:03:59.793165 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:03:59.793165 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 16:03:59.793165 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 16:03:59.793165 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 19 16:03:59.793165 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:03:59.793165 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 651/1298] 10.222.0.57 () {56 vars in 1117 bytes} [Mon Sep 19 16:03:59 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:03:59.797505 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c6d576-255b-4835-b1fe-4d240827600e None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:59.817471 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:03:59.900606 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c6d576-255b-4835-b1fe-4d240827600e None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:03:59.935607 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f4e134-4bc2-4360-a445-92e6166b6a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:03:59.936212 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f4e134-4bc2-4360-a445-92e6166b6a1b None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:03:59.936492 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f4e134-4bc2-4360-a445-92e6166b6a1b None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:03:59.936767 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f4e134-4bc2-4360-a445-92e6166b6a1b None admin] PATH_INFO: `/v3/services` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:03:59.937455 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f4e134-4bc2-4360-a445-92e6166b6a1b None admin] RBAC: Adding query filter params (name=placement) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:03:59.938004 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f4e134-4bc2-4360-a445-92e6166b6a1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:03:59.939405 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f4e134-4bc2-4360-a445-92e6166b6a1b None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:03:59.946874 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 648/1299] 10.222.0.57 () {56 vars in 1116 bytes} [Mon Sep 19 16:03:59 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:03:59.952875 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eefa00a-1154-4ea2-84a3-1b141d6f7e42 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:03:59.984244 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:04:00.056487 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eefa00a-1154-4ea2-84a3-1b141d6f7e42 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:04:00.089883 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd42646-9c76-4af5-96a9-8658e1857ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:04:00.090531 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd42646-9c76-4af5-96a9-8658e1857ba9 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:04:00.090810 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd42646-9c76-4af5-96a9-8658e1857ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:04:00.091082 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd42646-9c76-4af5-96a9-8658e1857ba9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:04:00.091775 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd42646-9c76-4af5-96a9-8658e1857ba9 None admin] RBAC: Adding query filter params (interface=public, service_id=2d0ca782920b43a9afb4dd1d7c3b303e, region_id=RegionOne) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:04:00.092077 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd42646-9c76-4af5-96a9-8658e1857ba9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:04:00.093433 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd42646-9c76-4af5-96a9-8658e1857ba9 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:04:00.098535 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 652/1300] 10.222.0.57 () {56 vars in 1251 bytes} [Mon Sep 19 16:03:59 2022] GET /identity/v3/endpoints?service_id=2d0ca782920b43a9afb4dd1d7c3b303e&interface=public®ion_id=RegionOne => generated 194 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:04:00.104374 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f4e134-4bc2-4360-a445-92e6166b6a1b None admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:04:00.187218 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:04:00.206497 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f4e134-4bc2-4360-a445-92e6166b6a1b None admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:04:00.239377 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19f936b-bfcb-448e-8a46-903985573477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:04:00.239979 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f936b-bfcb-448e-8a46-903985573477 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:04:00.240262 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f936b-bfcb-448e-8a46-903985573477 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:04:00.240534 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f936b-bfcb-448e-8a46-903985573477 None admin] PATH_INFO: `/v3/services` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:04:00.241127 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19f936b-bfcb-448e-8a46-903985573477 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:04:00.242503 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19f936b-bfcb-448e-8a46-903985573477 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:04:00.249194 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 649/1301] 10.222.0.57 () {56 vars in 1087 bytes} [Mon Sep 19 16:04:00 2022] GET /identity/v3/services => generated 2191 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:04:02.412556 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcd1e75-0921-42b6-9880-30b5a2d4c455 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:04:02.413140 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcd1e75-0921-42b6-9880-30b5a2d4c455 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:04:02.413140 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcd1e75-0921-42b6-9880-30b5a2d4c455 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:04:02.415146 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 653/1302] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:04:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:04:02.420844 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352351dc-db7b-48e8-9cac-bd93c0a002b0 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:04:02.421262 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352351dc-db7b-48e8-9cac-bd93c0a002b0 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:04:02.421384 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352351dc-db7b-48e8-9cac-bd93c0a002b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:04:02.439872 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-352351dc-db7b-48e8-9cac-bd93c0a002b0 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:04:02.441050 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-352351dc-db7b-48e8-9cac-bd93c0a002b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:04:02.507328 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 650/1303] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:04:02 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 16:04:02.776035 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcd1e75-0921-42b6-9880-30b5a2d4c455 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:04:02.885489 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:04:02.893285 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcd1e75-0921-42b6-9880-30b5a2d4c455 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:04:02.928399 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea916a86-70c0-4928-85ea-13b5d00f32e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:04:02.929053 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea916a86-70c0-4928-85ea-13b5d00f32e0 None admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:04:02.929332 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea916a86-70c0-4928-85ea-13b5d00f32e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:04:02.929609 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea916a86-70c0-4928-85ea-13b5d00f32e0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:04:02.930188 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea916a86-70c0-4928-85ea-13b5d00f32e0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:04:02.931596 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea916a86-70c0-4928-85ea-13b5d00f32e0 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:04:02.940491 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application [None req-ea916a86-70c0-4928-85ea-13b5d00f32e0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 19 16:04:02.940491 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:04:02.940491 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 16:04:02.940491 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 16:04:02.940491 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 16:04:02.940491 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 16:04:02.940491 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 16:04:02.940491 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 16:04:02.940491 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 19 16:04:02.940491 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:04:02.940491 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 16:04:02.940491 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:04:02.940491 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:04:02.940491 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:04:02.940491 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:04:02.940491 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:04:02.940491 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:04:02.942463 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:04:02.942463 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:04:02.942463 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:04:02.942463 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:04:02.942463 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:04:02.942463 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:04:02.942463 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 16:04:02.942463 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 16:04:02.942463 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 16:04:02.942463 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 16:04:02.942463 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:04:02.942463 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:04:02.942463 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:04:02.942463 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:04:02.942463 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:04:02.942463 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:04:02.943560 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:04:02.943560 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application with Lock( Sep 19 16:04:02.943560 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:04:02.943560 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:04:02.943560 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:04:02.943560 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:04:02.943560 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:04:02.943560 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:04:02.943560 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:04:02.943560 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:04:02.943560 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 16:04:02.943560 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 16:04:02.943560 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 19 16:04:02.943560 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: ERROR keystone.server.flask.application Sep 19 16:04:02.943560 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 654/1304] 10.222.0.57 () {56 vars in 1117 bytes} [Mon Sep 19 16:04:02 2022] GET /identity/v3/services/placement => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 16:04:02.948211 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352351dc-db7b-48e8-9cac-bd93c0a002b0 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:04:03.035997 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:04:03.061140 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352351dc-db7b-48e8-9cac-bd93c0a002b0 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:04:03.096473 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c12f388-05a0-4193-abc8-d694f5ee5ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:04:03.097078 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12f388-05a0-4193-abc8-d694f5ee5ba8 None admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:04:03.097358 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12f388-05a0-4193-abc8-d694f5ee5ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:04:03.097629 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12f388-05a0-4193-abc8-d694f5ee5ba8 None admin] PATH_INFO: `/v3/services` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:04:03.098254 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c12f388-05a0-4193-abc8-d694f5ee5ba8 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:04:03.098715 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c12f388-05a0-4193-abc8-d694f5ee5ba8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:04:03.100066 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c12f388-05a0-4193-abc8-d694f5ee5ba8 None admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:04:03.105091 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 651/1305] 10.222.0.57 () {56 vars in 1116 bytes} [Mon Sep 19 16:04:02 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:04:03.111074 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea916a86-70c0-4928-85ea-13b5d00f32e0 None admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:04:03.208239 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea916a86-70c0-4928-85ea-13b5d00f32e0 None admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:04:03.240809 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948bea08-1bfa-4795-8216-6bab7bce9650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:04:03.241453 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948bea08-1bfa-4795-8216-6bab7bce9650 None admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:04:03.241734 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948bea08-1bfa-4795-8216-6bab7bce9650 None admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:04:03.242009 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948bea08-1bfa-4795-8216-6bab7bce9650 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:04:03.242715 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948bea08-1bfa-4795-8216-6bab7bce9650 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:04:03.244089 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948bea08-1bfa-4795-8216-6bab7bce9650 None admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:04:03.263941 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 655/1306] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:04:03 2022] POST /identity/v3/endpoints => generated 331 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:04:13.102221 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=30 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:04:13.259541 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:05:10.331022 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7e303-fee5-4f2f-abcd-355c60761f5d None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:05:10.331499 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7e303-fee5-4f2f-abcd-355c60761f5d None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:05:10.331596 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7e303-fee5-4f2f-abcd-355c60761f5d None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:05:10.338309 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 652/1307] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:05:10 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:05:10.340296 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22941a27-d6e5-4230-ab90-2c8358f5f337 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:05:10.340886 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22941a27-d6e5-4230-ab90-2c8358f5f337 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:05:10.341318 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22941a27-d6e5-4230-ab90-2c8358f5f337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:05:10.487454 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-22941a27-d6e5-4230-ab90-2c8358f5f337 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:05:10.488740 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-22941a27-d6e5-4230-ab90-2c8358f5f337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:05:10.606693 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 656/1308] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:05:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:05:10.902331 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d0915-5ed4-4a38-b488-b9e407b69733 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:05:10.902766 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d0915-5ed4-4a38-b488-b9e407b69733 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:05:10.902984 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d0915-5ed4-4a38-b488-b9e407b69733 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:05:10.904595 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 653/1309] 10.222.0.57 () {54 vars in 963 bytes} [Mon Sep 19 16:05:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:05:10.911219 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc19f7f-e356-4c08-9a69-693dec3cc906 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:05:10.911573 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc19f7f-e356-4c08-9a69-693dec3cc906 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:05:10.911821 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc19f7f-e356-4c08-9a69-693dec3cc906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:05:11.003888 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-3cc19f7f-e356-4c08-9a69-693dec3cc906 None None] MFA Rules not processed for user `50355d7b1c6c4f0bbbbc7fe56efb9039`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:05:11.005691 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-3cc19f7f-e356-4c08-9a69-693dec3cc906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:05:11.039364 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:05:11.104105 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 657/1310] 10.222.0.57 () {58 vars in 1060 bytes} [Mon Sep 19 16:05:10 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:05:11.113318 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6d0915-5ed4-4a38-b488-b9e407b69733 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:05:11.115411 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-ac6d0915-5ed4-4a38-b488-b9e407b69733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:05:11.235429 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6d0915-5ed4-4a38-b488-b9e407b69733 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:05:11.321331 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b24faa3-ecfd-4a12-875c-4646bd4b82e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:05:11.321936 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b24faa3-ecfd-4a12-875c-4646bd4b82e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:05:11.322210 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b24faa3-ecfd-4a12-875c-4646bd4b82e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:05:11.322508 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b24faa3-ecfd-4a12-875c-4646bd4b82e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:05:11.324087 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-1b24faa3-ecfd-4a12-875c-4646bd4b82e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:05:11.384330 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:05:11.443898 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b24faa3-ecfd-4a12-875c-4646bd4b82e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:05:11.445374 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b24faa3-ecfd-4a12-875c-4646bd4b82e7 service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:05:11.468768 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 654/1311] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:05:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:05:13.983098 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c3ed9d-db94-4dd7-8af2-9448c5d5b62d None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:05:13.983746 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c3ed9d-db94-4dd7-8af2-9448c5d5b62d None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:05:13.983847 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c3ed9d-db94-4dd7-8af2-9448c5d5b62d None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:05:13.985383 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 658/1312] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:05:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:05:13.991187 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe418dab-9790-497b-8059-1352f0530fb7 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:05:13.991718 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe418dab-9790-497b-8059-1352f0530fb7 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:05:13.991718 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe418dab-9790-497b-8059-1352f0530fb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:05:14.085595 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-fe418dab-9790-497b-8059-1352f0530fb7 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:05:14.086948 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-fe418dab-9790-497b-8059-1352f0530fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:05:14.204814 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 655/1313] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:05:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:05:14.490716 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c3ed9d-db94-4dd7-8af2-9448c5d5b62d None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:05:14.492346 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-62c3ed9d-db94-4dd7-8af2-9448c5d5b62d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:05:14.605869 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c3ed9d-db94-4dd7-8af2-9448c5d5b62d None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:05:14.680763 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:05:14.683328 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8cfb9f-dbc4-400e-a1f2-443eb5536c0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:05:14.684112 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8cfb9f-dbc4-400e-a1f2-443eb5536c0b service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:05:14.684501 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8cfb9f-dbc4-400e-a1f2-443eb5536c0b service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:05:14.684924 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8cfb9f-dbc4-400e-a1f2-443eb5536c0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:05:14.686845 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-ab8cfb9f-dbc4-400e-a1f2-443eb5536c0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:05:14.805849 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8cfb9f-dbc4-400e-a1f2-443eb5536c0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:05:14.807330 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8cfb9f-dbc4-400e-a1f2-443eb5536c0b service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:05:14.826283 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 659/1314] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:05:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:05:17.354665 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c218d39d-0703-4b78-b904-6e9cf24784ef None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:05:17.355200 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c218d39d-0703-4b78-b904-6e9cf24784ef None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:05:17.355200 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c218d39d-0703-4b78-b904-6e9cf24784ef None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:05:17.356525 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 656/1315] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:05:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:05:17.361831 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558176e2-6322-428e-b25d-70602fa3af35 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:05:17.362286 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558176e2-6322-428e-b25d-70602fa3af35 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:05:17.362403 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558176e2-6322-428e-b25d-70602fa3af35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:05:17.429921 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-558176e2-6322-428e-b25d-70602fa3af35 None None] MFA Rules not processed for user `bbf2a95469bc4f3593466124dfc85117`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:05:17.431292 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-558176e2-6322-428e-b25d-70602fa3af35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:05:17.484368 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:05:17.529150 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 660/1316] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:05:17 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:05:17.832151 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a8c79-fc66-4d8d-8816-1e6aa251b657 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:05:17.832433 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a8c79-fc66-4d8d-8816-1e6aa251b657 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:05:17.832707 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a8c79-fc66-4d8d-8816-1e6aa251b657 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:05:17.834107 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 657/1317] 10.222.0.57 () {54 vars in 963 bytes} [Mon Sep 19 16:05:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:05:17.840404 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a72210-e58f-4196-ae25-000bdc31e92f None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:05:17.840692 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a72210-e58f-4196-ae25-000bdc31e92f None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:05:17.840971 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a72210-e58f-4196-ae25-000bdc31e92f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:05:17.920246 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-a9a72210-e58f-4196-ae25-000bdc31e92f None None] MFA Rules not processed for user `50355d7b1c6c4f0bbbbc7fe56efb9039`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:05:17.921469 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-a9a72210-e58f-4196-ae25-000bdc31e92f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:05:18.004226 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 661/1318] 10.222.0.57 () {58 vars in 1060 bytes} [Mon Sep 19 16:05:17 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:05:18.013689 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768a8c79-fc66-4d8d-8816-1e6aa251b657 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:05:18.016230 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-768a8c79-fc66-4d8d-8816-1e6aa251b657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:05:18.032226 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:05:18.133917 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768a8c79-fc66-4d8d-8816-1e6aa251b657 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:05:18.212787 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafeaba0-1dd6-4a3b-a640-d72778500db2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:05:18.213402 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafeaba0-1dd6-4a3b-a640-d72778500db2 service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:05:18.213678 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafeaba0-1dd6-4a3b-a640-d72778500db2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:05:18.213975 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafeaba0-1dd6-4a3b-a640-d72778500db2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:05:18.215637 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-dafeaba0-1dd6-4a3b-a640-d72778500db2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:05:18.279151 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:05:18.326204 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafeaba0-1dd6-4a3b-a640-d72778500db2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:05:18.327660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafeaba0-1dd6-4a3b-a640-d72778500db2 service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:05:18.347844 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 658/1319] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:05:18 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:05:21.543086 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43143d50-fb3f-4ef3-a39f-8ea80c8918f4 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:05:21.543882 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43143d50-fb3f-4ef3-a39f-8ea80c8918f4 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:05:21.543882 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43143d50-fb3f-4ef3-a39f-8ea80c8918f4 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:05:21.545655 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 662/1320] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:05:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:05:21.551142 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4775b-8124-4089-a006-661c6e2af304 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:05:21.551577 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4775b-8124-4089-a006-661c6e2af304 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:05:21.551767 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4775b-8124-4089-a006-661c6e2af304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:05:21.623192 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-01a4775b-8124-4089-a006-661c6e2af304 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:05:21.624409 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-01a4775b-8124-4089-a006-661c6e2af304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:05:21.734995 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 659/1321] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:05:21 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:05:22.026494 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43143d50-fb3f-4ef3-a39f-8ea80c8918f4 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:05:22.028091 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-43143d50-fb3f-4ef3-a39f-8ea80c8918f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:05:22.086358 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:05:22.150768 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43143d50-fb3f-4ef3-a39f-8ea80c8918f4 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:05:22.237064 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2981943b-86dd-4b24-8759-761a3fb66720 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:05:22.237683 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2981943b-86dd-4b24-8759-761a3fb66720 service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:05:22.237965 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2981943b-86dd-4b24-8759-761a3fb66720 service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:05:22.238236 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2981943b-86dd-4b24-8759-761a3fb66720 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:05:22.240010 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-2981943b-86dd-4b24-8759-761a3fb66720 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:05:22.357048 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:05:22.371958 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2981943b-86dd-4b24-8759-761a3fb66720 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:05:22.373440 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2981943b-86dd-4b24-8759-761a3fb66720 service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:05:22.397108 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 663/1322] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:05:22 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:05:25.339038 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a0aff-940a-465a-9d31-65c55e415646 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:05:25.339638 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a0aff-940a-465a-9d31-65c55e415646 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:05:25.339638 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a0aff-940a-465a-9d31-65c55e415646 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:05:25.340929 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 660/1323] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:05:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:05:25.346428 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d25298-9112-47a0-ba27-f963b1bcada2 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:05:25.346939 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d25298-9112-47a0-ba27-f963b1bcada2 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:05:25.347023 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d25298-9112-47a0-ba27-f963b1bcada2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:05:25.425779 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-31d25298-9112-47a0-ba27-f963b1bcada2 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:05:25.427033 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-31d25298-9112-47a0-ba27-f963b1bcada2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:05:25.535527 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 664/1324] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:05:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:05:25.817058 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4a0aff-940a-465a-9d31-65c55e415646 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:05:25.819224 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-7a4a0aff-940a-465a-9d31-65c55e415646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:05:25.840274 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:05:25.931444 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4a0aff-940a-465a-9d31-65c55e415646 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:05:26.010463 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db9dfb7-eb2f-4cae-b06d-f39a0e98b961 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:05:26.011220 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db9dfb7-eb2f-4cae-b06d-f39a0e98b961 service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:05:26.011589 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db9dfb7-eb2f-4cae-b06d-f39a0e98b961 service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:05:26.011961 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db9dfb7-eb2f-4cae-b06d-f39a0e98b961 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:05:26.013728 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-8db9dfb7-eb2f-4cae-b06d-f39a0e98b961 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:05:26.090657 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:05:26.131171 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db9dfb7-eb2f-4cae-b06d-f39a0e98b961 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:05:26.132611 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db9dfb7-eb2f-4cae-b06d-f39a0e98b961 service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:05:26.152229 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 661/1325] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:05:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:05:28.411360 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e4bca2-3a86-46de-a268-aa28f7b0adb2 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:05:28.411902 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e4bca2-3a86-46de-a268-aa28f7b0adb2 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:05:28.411902 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e4bca2-3a86-46de-a268-aa28f7b0adb2 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:05:28.413267 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 665/1326] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:05:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:05:28.419026 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e258a-1661-49c9-b8f6-5c11d3e667e8 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:05:28.419823 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e258a-1661-49c9-b8f6-5c11d3e667e8 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:05:28.419958 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e258a-1661-49c9-b8f6-5c11d3e667e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:05:28.493198 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-516e258a-1661-49c9-b8f6-5c11d3e667e8 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:05:28.494438 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-516e258a-1661-49c9-b8f6-5c11d3e667e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:05:28.615105 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 662/1327] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:05:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:05:28.901057 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e4bca2-3a86-46de-a268-aa28f7b0adb2 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:05:28.902709 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-17e4bca2-3a86-46de-a268-aa28f7b0adb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:05:28.966759 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:05:29.013795 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e4bca2-3a86-46de-a268-aa28f7b0adb2 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:05:29.093612 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f1e27f-96ea-4015-9d23-f040d69838f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:05:29.094217 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f1e27f-96ea-4015-9d23-f040d69838f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:05:29.094527 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f1e27f-96ea-4015-9d23-f040d69838f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:05:29.094796 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f1e27f-96ea-4015-9d23-f040d69838f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:05:29.096403 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-a0f1e27f-96ea-4015-9d23-f040d69838f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:05:29.212486 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:05:29.215330 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f1e27f-96ea-4015-9d23-f040d69838f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:05:29.216932 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f1e27f-96ea-4015-9d23-f040d69838f6 service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:05:29.239946 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 666/1328] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:05:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:05:31.575551 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a6233d-fe73-49aa-b418-35a963f6f39e None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:05:31.576056 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a6233d-fe73-49aa-b418-35a963f6f39e None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:05:31.576175 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a6233d-fe73-49aa-b418-35a963f6f39e None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:05:31.577622 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 663/1329] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:05:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:05:31.583727 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260b2d35-ef61-4f3b-b304-f482b4b5ae01 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:05:31.584295 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260b2d35-ef61-4f3b-b304-f482b4b5ae01 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:05:31.584295 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260b2d35-ef61-4f3b-b304-f482b4b5ae01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:05:31.673835 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-260b2d35-ef61-4f3b-b304-f482b4b5ae01 None None] MFA Rules not processed for user `bbf2a95469bc4f3593466124dfc85117`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:05:31.675110 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-260b2d35-ef61-4f3b-b304-f482b4b5ae01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:05:31.786240 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 667/1330] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:05:31 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:05:32.070624 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a6233d-fe73-49aa-b418-35a963f6f39e None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:05:32.072281 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-37a6233d-fe73-49aa-b418-35a963f6f39e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:05:32.107048 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:05:32.189371 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a6233d-fe73-49aa-b418-35a963f6f39e None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:05:32.271996 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa811f92-2ebc-4f77-abe9-16c41b6e619b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:05:32.272601 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa811f92-2ebc-4f77-abe9-16c41b6e619b service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:05:32.272880 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa811f92-2ebc-4f77-abe9-16c41b6e619b service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:05:32.273150 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa811f92-2ebc-4f77-abe9-16c41b6e619b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:05:32.274849 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-fa811f92-2ebc-4f77-abe9-16c41b6e619b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:05:32.360613 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:05:32.394556 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa811f92-2ebc-4f77-abe9-16c41b6e619b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:05:32.396538 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa811f92-2ebc-4f77-abe9-16c41b6e619b service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:05:32.416579 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 664/1331] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:05:32 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:05:34.923604 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d36d653-13b0-4a0a-918d-f45ddcf94913 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:05:34.924208 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d36d653-13b0-4a0a-918d-f45ddcf94913 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:05:34.924208 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d36d653-13b0-4a0a-918d-f45ddcf94913 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:05:34.925499 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 668/1332] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:05:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:05:34.932082 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00852956-292c-4818-aad8-18e8aed26565 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:05:34.932636 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00852956-292c-4818-aad8-18e8aed26565 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:05:34.932636 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00852956-292c-4818-aad8-18e8aed26565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:05:35.015676 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-00852956-292c-4818-aad8-18e8aed26565 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:05:35.016972 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-00852956-292c-4818-aad8-18e8aed26565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:05:35.127528 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 665/1333] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:05:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:05:35.416112 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d36d653-13b0-4a0a-918d-f45ddcf94913 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:05:35.417709 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-6d36d653-13b0-4a0a-918d-f45ddcf94913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:05:35.507240 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:05:35.547271 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d36d653-13b0-4a0a-918d-f45ddcf94913 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:05:35.649481 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc4b453-6529-4d64-8667-4bdcf457de7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:05:35.650162 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4b453-6529-4d64-8667-4bdcf457de7d service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:05:35.650519 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4b453-6529-4d64-8667-4bdcf457de7d service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:05:35.650841 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4b453-6529-4d64-8667-4bdcf457de7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:05:35.652616 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-5bc4b453-6529-4d64-8667-4bdcf457de7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:05:35.786888 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc4b453-6529-4d64-8667-4bdcf457de7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:05:35.788477 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc4b453-6529-4d64-8667-4bdcf457de7d service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:05:35.799952 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:05:35.811551 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 669/1334] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:05:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 16:05:38.178560 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e78800-28a0-4b5d-8f6b-c9ce9825250d None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:05:38.179145 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e78800-28a0-4b5d-8f6b-c9ce9825250d None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:05:38.179145 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e78800-28a0-4b5d-8f6b-c9ce9825250d None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:05:38.180542 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 666/1335] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:05:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 16:05:38.186631 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed994f74-0b28-4df4-8aa4-bf3ed72f69f9 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:05:38.187476 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed994f74-0b28-4df4-8aa4-bf3ed72f69f9 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:05:38.187476 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed994f74-0b28-4df4-8aa4-bf3ed72f69f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:05:38.292414 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-ed994f74-0b28-4df4-8aa4-bf3ed72f69f9 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:05:38.294399 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-ed994f74-0b28-4df4-8aa4-bf3ed72f69f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:05:38.489511 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 670/1336] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:05:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:05:38.785135 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e78800-28a0-4b5d-8f6b-c9ce9825250d None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:05:38.788689 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-00e78800-28a0-4b5d-8f6b-c9ce9825250d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:05:38.860959 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:05:38.939391 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e78800-28a0-4b5d-8f6b-c9ce9825250d None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:05:39.020189 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde99000-3ed6-46a1-89ca-b18dc41685c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:05:39.020810 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde99000-3ed6-46a1-89ca-b18dc41685c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:05:39.021091 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde99000-3ed6-46a1-89ca-b18dc41685c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:05:39.021367 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde99000-3ed6-46a1-89ca-b18dc41685c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:05:39.023165 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-fde99000-3ed6-46a1-89ca-b18dc41685c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:05:39.115547 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:05:39.142900 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde99000-3ed6-46a1-89ca-b18dc41685c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:05:39.144346 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde99000-3ed6-46a1-89ca-b18dc41685c9 service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:05:39.162850 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 667/1337] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:05:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:05:41.924317 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a4b9b9-0068-4ab1-bf71-f2995b4ade0e None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:05:41.924842 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a4b9b9-0068-4ab1-bf71-f2995b4ade0e None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:05:41.924842 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a4b9b9-0068-4ab1-bf71-f2995b4ade0e None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:05:41.926126 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 671/1338] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:05:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:05:41.931649 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdad55a-9634-48a5-bab1-d0db27adf928 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:05:41.932085 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdad55a-9634-48a5-bab1-d0db27adf928 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:05:41.932209 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdad55a-9634-48a5-bab1-d0db27adf928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:05:42.003779 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-0cdad55a-9634-48a5-bab1-d0db27adf928 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:05:42.005608 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-0cdad55a-9634-48a5-bab1-d0db27adf928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:05:42.105419 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 668/1339] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:05:41 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:05:42.388211 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a4b9b9-0068-4ab1-bf71-f2995b4ade0e None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:05:42.389752 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-e9a4b9b9-0068-4ab1-bf71-f2995b4ade0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:05:42.466685 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:05:42.494429 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a4b9b9-0068-4ab1-bf71-f2995b4ade0e None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:05:42.578665 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e89467b-e713-408a-8ab0-b9b2080c8bcf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:05:42.579293 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e89467b-e713-408a-8ab0-b9b2080c8bcf service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:05:42.579574 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e89467b-e713-408a-8ab0-b9b2080c8bcf service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:05:42.579849 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e89467b-e713-408a-8ab0-b9b2080c8bcf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:05:42.581576 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-6e89467b-e713-408a-8ab0-b9b2080c8bcf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:05:42.707330 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e89467b-e713-408a-8ab0-b9b2080c8bcf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:05:42.709429 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e89467b-e713-408a-8ab0-b9b2080c8bcf service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:05:42.730068 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 672/1340] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:05:42 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:05:47.012871 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d286128-2574-4496-8611-577beefa5eb8 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:05:47.013504 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d286128-2574-4496-8611-577beefa5eb8 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:05:47.013504 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d286128-2574-4496-8611-577beefa5eb8 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:05:47.015113 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 669/1341] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:05:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:05:47.020879 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea9dac4-714c-4a0c-965d-de8aae7ea2e7 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:05:47.021377 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea9dac4-714c-4a0c-965d-de8aae7ea2e7 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:05:47.021481 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea9dac4-714c-4a0c-965d-de8aae7ea2e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:05:47.027299 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:05:47.100252 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-8ea9dac4-714c-4a0c-965d-de8aae7ea2e7 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:05:47.101443 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-8ea9dac4-714c-4a0c-965d-de8aae7ea2e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:05:47.206008 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 673/1342] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:05:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:05:47.495157 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d286128-2574-4496-8611-577beefa5eb8 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:05:47.497186 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-3d286128-2574-4496-8611-577beefa5eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:05:47.549059 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:05:47.606617 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d286128-2574-4496-8611-577beefa5eb8 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:05:47.681654 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799366d0-dbb0-4c4c-96c7-a8ab4448e7f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:05:47.682260 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799366d0-dbb0-4c4c-96c7-a8ab4448e7f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:05:47.682566 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799366d0-dbb0-4c4c-96c7-a8ab4448e7f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:05:47.682841 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799366d0-dbb0-4c4c-96c7-a8ab4448e7f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:05:47.684449 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-799366d0-dbb0-4c4c-96c7-a8ab4448e7f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:05:47.784397 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:05:47.803712 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799366d0-dbb0-4c4c-96c7-a8ab4448e7f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:05:47.805069 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799366d0-dbb0-4c4c-96c7-a8ab4448e7f6 service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:05:47.823642 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 670/1343] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:05:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:05:50.694784 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7643472c-6242-4b84-a07e-871e2ac5b298 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:05:50.695340 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7643472c-6242-4b84-a07e-871e2ac5b298 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:05:50.695340 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7643472c-6242-4b84-a07e-871e2ac5b298 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:05:50.696670 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 674/1344] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:05:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:05:50.702043 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a7365-868d-4c46-8358-2ee93dd38015 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:05:50.702471 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a7365-868d-4c46-8358-2ee93dd38015 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:05:50.702633 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a7365-868d-4c46-8358-2ee93dd38015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:05:50.780018 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-db9a7365-868d-4c46-8358-2ee93dd38015 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:05:50.781261 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-db9a7365-868d-4c46-8358-2ee93dd38015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:05:50.895269 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 671/1345] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:05:50 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:05:51.188224 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7643472c-6242-4b84-a07e-871e2ac5b298 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:05:51.189830 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-7643472c-6242-4b84-a07e-871e2ac5b298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:05:51.286104 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:05:51.306808 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7643472c-6242-4b84-a07e-871e2ac5b298 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:05:51.389820 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636e1404-2b80-43f4-9602-573e9199f163 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:05:51.390339 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e1404-2b80-43f4-9602-573e9199f163 service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:05:51.390644 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e1404-2b80-43f4-9602-573e9199f163 service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:05:51.390917 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e1404-2b80-43f4-9602-573e9199f163 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:05:51.392533 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-636e1404-2b80-43f4-9602-573e9199f163 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:05:51.507189 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636e1404-2b80-43f4-9602-573e9199f163 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:05:51.508627 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636e1404-2b80-43f4-9602-573e9199f163 service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:05:51.527119 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 675/1346] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:05:51 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:05:56.087858 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb939252-95c1-4505-bbaf-f2af1e6604ad None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:05:56.088588 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb939252-95c1-4505-bbaf-f2af1e6604ad None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:05:56.088720 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb939252-95c1-4505-bbaf-f2af1e6604ad None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:05:56.091161 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 672/1347] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:05:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:05:56.097868 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8fef25-d946-4913-8408-ea2f6b0aaf6b None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:05:56.098774 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8fef25-d946-4913-8408-ea2f6b0aaf6b None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:05:56.098938 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8fef25-d946-4913-8408-ea2f6b0aaf6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:05:56.115455 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:05:56.169842 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-5d8fef25-d946-4913-8408-ea2f6b0aaf6b None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:05:56.171363 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-5d8fef25-d946-4913-8408-ea2f6b0aaf6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:05:56.268423 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 676/1348] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:05:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:05:56.568062 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb939252-95c1-4505-bbaf-f2af1e6604ad None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:05:56.569631 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-cb939252-95c1-4505-bbaf-f2af1e6604ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:05:56.632682 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:05:56.683801 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb939252-95c1-4505-bbaf-f2af1e6604ad None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:05:56.765002 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f0974c-4395-4fc4-a4a6-55264c101df3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:05:56.765660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f0974c-4395-4fc4-a4a6-55264c101df3 service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:05:56.765955 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f0974c-4395-4fc4-a4a6-55264c101df3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:05:56.766300 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f0974c-4395-4fc4-a4a6-55264c101df3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:05:56.768109 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-57f0974c-4395-4fc4-a4a6-55264c101df3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:05:56.880126 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:05:56.884073 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f0974c-4395-4fc4-a4a6-55264c101df3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:05:56.885506 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f0974c-4395-4fc4-a4a6-55264c101df3 service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:05:56.903767 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 673/1349] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:05:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 16:06:01.960027 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e03afd-6ed2-42bd-b63d-5519dbb6b021 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:06:01.960510 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e03afd-6ed2-42bd-b63d-5519dbb6b021 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:06:01.960947 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e03afd-6ed2-42bd-b63d-5519dbb6b021 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:06:01.963418 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 677/1350] 10.222.0.57 () {54 vars in 921 bytes} [Mon Sep 19 16:06:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:06:01.970990 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa89ae7-1a4d-4e33-a97e-07ac41f946c7 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:06:01.971705 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa89ae7-1a4d-4e33-a97e-07ac41f946c7 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:06:01.971705 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa89ae7-1a4d-4e33-a97e-07ac41f946c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:06:02.082700 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-dfa89ae7-1a4d-4e33-a97e-07ac41f946c7 None None] MFA Rules not processed for user `1f1a985495ed48a4acc48e2e6fc7f977`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:06:02.083232 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-dfa89ae7-1a4d-4e33-a97e-07ac41f946c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:06:02.277085 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 674/1351] 10.222.0.57 () {58 vars in 1018 bytes} [Mon Sep 19 16:06:01 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:06:06.266858 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=59 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:06:06.448364 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb14d561-09f1-4e52-8f17-ccdcb3b4a645 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:06:06.448769 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb14d561-09f1-4e52-8f17-ccdcb3b4a645 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:06:06.449248 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb14d561-09f1-4e52-8f17-ccdcb3b4a645 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:06:06.450892 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 678/1352] 10.222.0.57 () {54 vars in 921 bytes} [Mon Sep 19 16:06:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:06:06.456205 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7e33d7-51c7-4b7e-9179-30ddbbe99a8a None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:06:06.456707 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7e33d7-51c7-4b7e-9179-30ddbbe99a8a None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:06:06.456788 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7e33d7-51c7-4b7e-9179-30ddbbe99a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:06:06.515161 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:06:06.541323 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-cf7e33d7-51c7-4b7e-9179-30ddbbe99a8a None None] MFA Rules not processed for user `1f1a985495ed48a4acc48e2e6fc7f977`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:06:06.542566 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-cf7e33d7-51c7-4b7e-9179-30ddbbe99a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:06:06.670724 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 675/1353] 10.222.0.57 () {58 vars in 1018 bytes} [Mon Sep 19 16:06:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:06:08.191230 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ed19a-b5fb-4d1c-9af8-412fbceeef81 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:06:08.191740 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ed19a-b5fb-4d1c-9af8-412fbceeef81 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:06:08.191740 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ed19a-b5fb-4d1c-9af8-412fbceeef81 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:06:08.193283 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 679/1354] 10.222.0.57 () {54 vars in 921 bytes} [Mon Sep 19 16:06:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:06:08.198666 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e139d5d-dcdf-4f5e-aab9-c2d30fe4f833 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:06:08.199150 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e139d5d-dcdf-4f5e-aab9-c2d30fe4f833 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:06:08.199255 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e139d5d-dcdf-4f5e-aab9-c2d30fe4f833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:06:08.280502 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-2e139d5d-dcdf-4f5e-aab9-c2d30fe4f833 None None] MFA Rules not processed for user `1f1a985495ed48a4acc48e2e6fc7f977`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:06:08.281717 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-2e139d5d-dcdf-4f5e-aab9-c2d30fe4f833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:06:08.327783 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:06:08.399955 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 676/1355] 10.222.0.57 () {58 vars in 1018 bytes} [Mon Sep 19 16:06:08 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:06:12.208817 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07d114-902f-4f3e-af61-f8c4f596edbb None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:06:12.209681 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07d114-902f-4f3e-af61-f8c4f596edbb None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:06:12.210110 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07d114-902f-4f3e-af61-f8c4f596edbb None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:06:12.212151 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 680/1356] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:06:12 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:06:12.218897 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a76eb-2c2b-4d74-a765-2611987e8461 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:06:12.219835 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a76eb-2c2b-4d74-a765-2611987e8461 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:06:12.220310 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a76eb-2c2b-4d74-a765-2611987e8461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:06:12.301657 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-937a76eb-2c2b-4d74-a765-2611987e8461 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:06:12.303127 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-937a76eb-2c2b-4d74-a765-2611987e8461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:06:12.408940 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 677/1357] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:06:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:06:12.712774 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1076f2-d97a-494f-9b52-07571298d143 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:06:12.713289 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1076f2-d97a-494f-9b52-07571298d143 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:06:12.713788 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1076f2-d97a-494f-9b52-07571298d143 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:06:12.715536 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 681/1358] 10.222.0.57 () {54 vars in 965 bytes} [Mon Sep 19 16:06:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:06:12.728245 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7374a63d-dc48-4996-8a2e-b1ada634181f None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:06:12.728558 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7374a63d-dc48-4996-8a2e-b1ada634181f None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:06:12.729621 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7374a63d-dc48-4996-8a2e-b1ada634181f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:06:12.754690 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:06:12.834756 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-7374a63d-dc48-4996-8a2e-b1ada634181f None None] MFA Rules not processed for user `1ecd01cc72834b72b5ff2c4b47249737`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:06:12.835996 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-7374a63d-dc48-4996-8a2e-b1ada634181f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:06:12.996041 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 678/1359] 10.222.0.57 () {58 vars in 1062 bytes} [Mon Sep 19 16:06:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:06:13.004723 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1076f2-d97a-494f-9b52-07571298d143 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:06:13.008105 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-af1076f2-d97a-494f-9b52-07571298d143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:06:13.238642 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1076f2-d97a-494f-9b52-07571298d143 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:06:13.320010 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:06:13.370874 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa99e8f-d6ee-4c6e-993e-fecc3bee43d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:06:13.371774 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa99e8f-d6ee-4c6e-993e-fecc3bee43d5 service nova] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:06:13.372038 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa99e8f-d6ee-4c6e-993e-fecc3bee43d5 service nova] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:06:13.372306 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa99e8f-d6ee-4c6e-993e-fecc3bee43d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:06:13.373889 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-9fa99e8f-d6ee-4c6e-993e-fecc3bee43d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:06:13.481914 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa99e8f-d6ee-4c6e-993e-fecc3bee43d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:06:13.483705 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa99e8f-d6ee-4c6e-993e-fecc3bee43d5 service nova] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:06:13.504855 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 682/1360] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:06:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:06:15.959995 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3e4469-9996-4af6-a111-5de479578c2b None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:06:15.959995 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3e4469-9996-4af6-a111-5de479578c2b None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:06:15.959995 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3e4469-9996-4af6-a111-5de479578c2b None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:06:15.962293 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 679/1361] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:06:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 16:06:15.967284 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114cc553-83b9-4c13-a2ee-9a36d494ced6 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:06:15.967797 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114cc553-83b9-4c13-a2ee-9a36d494ced6 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:06:15.967894 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114cc553-83b9-4c13-a2ee-9a36d494ced6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:06:16.078653 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-114cc553-83b9-4c13-a2ee-9a36d494ced6 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:06:16.080592 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-114cc553-83b9-4c13-a2ee-9a36d494ced6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:06:16.148758 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:06:16.248006 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 683/1362] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:06:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:06:16.572089 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3e4469-9996-4af6-a111-5de479578c2b None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:06:16.573852 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-8a3e4469-9996-4af6-a111-5de479578c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:06:16.677683 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:06:16.796296 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3e4469-9996-4af6-a111-5de479578c2b None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:06:16.928116 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b7bf09-fa46-4b4e-95ff-22cfcb4b92b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:06:16.928997 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b7bf09-fa46-4b4e-95ff-22cfcb4b92b9 service nova] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:06:16.928997 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b7bf09-fa46-4b4e-95ff-22cfcb4b92b9 service nova] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:06:16.929194 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b7bf09-fa46-4b4e-95ff-22cfcb4b92b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:06:16.930880 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-99b7bf09-fa46-4b4e-95ff-22cfcb4b92b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:06:16.957690 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:06:17.045806 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b7bf09-fa46-4b4e-95ff-22cfcb4b92b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:06:17.047478 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b7bf09-fa46-4b4e-95ff-22cfcb4b92b9 service nova] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:06:17.070309 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 680/1363] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:06:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:06:19.739580 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94727f1f-cdd4-4725-b677-1090965a0a3f None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:06:19.740674 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94727f1f-cdd4-4725-b677-1090965a0a3f None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:06:19.740674 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94727f1f-cdd4-4725-b677-1090965a0a3f None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:06:19.741860 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 684/1364] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:06:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:06:19.748615 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae195d8-8af2-4473-a0dd-4282d015e291 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:06:19.749351 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae195d8-8af2-4473-a0dd-4282d015e291 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:06:19.749351 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae195d8-8af2-4473-a0dd-4282d015e291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:06:19.841043 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-8ae195d8-8af2-4473-a0dd-4282d015e291 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:06:19.842277 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-8ae195d8-8af2-4473-a0dd-4282d015e291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:06:19.955218 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:06:19.962429 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 681/1365] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:06:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:06:20.264336 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b86009a-a954-4f15-b590-a9b49b076d7a None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:06:20.264780 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b86009a-a954-4f15-b590-a9b49b076d7a None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:06:20.265153 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b86009a-a954-4f15-b590-a9b49b076d7a None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:06:20.266624 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 685/1366] 10.222.0.57 () {54 vars in 965 bytes} [Mon Sep 19 16:06:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:06:20.272524 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec07b19-9a10-4b00-9844-ad9fc699c12c None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:06:20.273825 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec07b19-9a10-4b00-9844-ad9fc699c12c None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:06:20.273825 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec07b19-9a10-4b00-9844-ad9fc699c12c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:06:20.370483 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-dec07b19-9a10-4b00-9844-ad9fc699c12c None None] MFA Rules not processed for user `1ecd01cc72834b72b5ff2c4b47249737`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:06:20.372031 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-dec07b19-9a10-4b00-9844-ad9fc699c12c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:06:20.503012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 682/1367] 10.222.0.57 () {58 vars in 1062 bytes} [Mon Sep 19 16:06:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:06:20.511459 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b86009a-a954-4f15-b590-a9b49b076d7a None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:06:20.513000 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-8b86009a-a954-4f15-b590-a9b49b076d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:06:20.716475 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b86009a-a954-4f15-b590-a9b49b076d7a None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:06:20.743046 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:06:20.752760 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46080f6-af00-43c5-b26e-c52883ea97db None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:06:20.753403 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46080f6-af00-43c5-b26e-c52883ea97db None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:06:20.753403 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46080f6-af00-43c5-b26e-c52883ea97db None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:06:20.754989 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 683/1368] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:06:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 16:06:20.762582 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e22ec16-777b-4e37-ad68-e6d899596057 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:06:20.762582 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e22ec16-777b-4e37-ad68-e6d899596057 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:06:20.762582 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e22ec16-777b-4e37-ad68-e6d899596057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:06:20.839227 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:06:20.851866 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-9e22ec16-777b-4e37-ad68-e6d899596057 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:06:20.853243 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-9e22ec16-777b-4e37-ad68-e6d899596057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:06:20.882460 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1702e5bc-bfa7-4ffa-bc1f-32aa9133dc55 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:06:20.883393 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1702e5bc-bfa7-4ffa-bc1f-32aa9133dc55 service nova] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:06:20.883666 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1702e5bc-bfa7-4ffa-bc1f-32aa9133dc55 service nova] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:06:20.883932 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1702e5bc-bfa7-4ffa-bc1f-32aa9133dc55 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:06:20.886558 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-1702e5bc-bfa7-4ffa-bc1f-32aa9133dc55 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:06:20.970527 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 684/1369] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:06:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:06:21.006828 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1702e5bc-bfa7-4ffa-bc1f-32aa9133dc55 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:06:21.008473 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1702e5bc-bfa7-4ffa-bc1f-32aa9133dc55 service nova] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:06:21.033780 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 686/1370] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:06:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:06:21.272527 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9fad8c-ff14-4af1-a969-23936f4b1a41 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:06:21.272957 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9fad8c-ff14-4af1-a969-23936f4b1a41 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:06:21.273428 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9fad8c-ff14-4af1-a969-23936f4b1a41 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:06:21.275164 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 685/1371] 10.222.0.57 () {54 vars in 962 bytes} [Mon Sep 19 16:06:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:06:21.281047 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e703e474-424c-4f90-9cfb-14f0c1e77af1 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:06:21.281475 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e703e474-424c-4f90-9cfb-14f0c1e77af1 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:06:21.281851 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e703e474-424c-4f90-9cfb-14f0c1e77af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:06:21.322599 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:06:21.382108 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-e703e474-424c-4f90-9cfb-14f0c1e77af1 None None] MFA Rules not processed for user `7d596e43cc7446d29e2d62d9052d6091`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:06:21.383386 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-e703e474-424c-4f90-9cfb-14f0c1e77af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:06:21.473974 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 687/1372] 10.222.0.57 () {58 vars in 1059 bytes} [Mon Sep 19 16:06:21 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:06:21.482221 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9fad8c-ff14-4af1-a969-23936f4b1a41 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:06:21.483937 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-3f9fad8c-ff14-4af1-a969-23936f4b1a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:06:21.600351 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9fad8c-ff14-4af1-a969-23936f4b1a41 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:06:21.640104 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:06:21.705293 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099f76e2-4c6d-45f4-bb43-88e3f6af7f32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:06:21.706079 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099f76e2-4c6d-45f4-bb43-88e3f6af7f32 service cinder] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:06:21.706560 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099f76e2-4c6d-45f4-bb43-88e3f6af7f32 service cinder] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:06:21.707466 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099f76e2-4c6d-45f4-bb43-88e3f6af7f32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:06:21.709413 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-099f76e2-4c6d-45f4-bb43-88e3f6af7f32 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:06:21.869710 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099f76e2-4c6d-45f4-bb43-88e3f6af7f32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:06:21.872466 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099f76e2-4c6d-45f4-bb43-88e3f6af7f32 service cinder] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:06:21.903344 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 686/1373] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:06:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:06:23.385230 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35510249-fa48-41b9-82da-2605c5c20dc0 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:06:23.385832 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35510249-fa48-41b9-82da-2605c5c20dc0 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:06:23.385832 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35510249-fa48-41b9-82da-2605c5c20dc0 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:06:23.387826 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 688/1374] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:06:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:06:23.393628 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37cb08f-ec77-4137-891e-def5263edfd7 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:06:23.394709 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37cb08f-ec77-4137-891e-def5263edfd7 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:06:23.394709 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37cb08f-ec77-4137-891e-def5263edfd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:06:23.493131 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-c37cb08f-ec77-4137-891e-def5263edfd7 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:06:23.494977 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-c37cb08f-ec77-4137-891e-def5263edfd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:06:23.507709 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:06:23.642999 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 687/1375] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:06:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:06:23.984699 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35510249-fa48-41b9-82da-2605c5c20dc0 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:06:23.986266 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-35510249-fa48-41b9-82da-2605c5c20dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:06:24.134764 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:06:24.177896 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35510249-fa48-41b9-82da-2605c5c20dc0 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:06:24.340640 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9818fa72-565e-40cf-93ea-852f28db06a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:06:24.340640 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9818fa72-565e-40cf-93ea-852f28db06a9 service nova] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:06:24.340640 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9818fa72-565e-40cf-93ea-852f28db06a9 service nova] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:06:24.340640 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9818fa72-565e-40cf-93ea-852f28db06a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:06:24.341973 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-9818fa72-565e-40cf-93ea-852f28db06a9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:06:24.443694 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:06:24.469235 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9818fa72-565e-40cf-93ea-852f28db06a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:06:24.472872 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9818fa72-565e-40cf-93ea-852f28db06a9 service nova] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:06:24.507695 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 689/1376] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:06:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:06:26.813792 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e527f69a-d4c7-4774-a618-422af58d23fc None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:06:26.814748 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e527f69a-d4c7-4774-a618-422af58d23fc None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:06:26.814748 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e527f69a-d4c7-4774-a618-422af58d23fc None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:06:26.816037 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 688/1377] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:06:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:06:26.821770 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d1d54-1999-4f3f-b7c6-235afcb4b3d9 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:06:26.822546 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d1d54-1999-4f3f-b7c6-235afcb4b3d9 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:06:26.822546 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d1d54-1999-4f3f-b7c6-235afcb4b3d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:06:26.901721 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-173d1d54-1999-4f3f-b7c6-235afcb4b3d9 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:06:26.903139 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-173d1d54-1999-4f3f-b7c6-235afcb4b3d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:06:27.052216 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 690/1378] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:06:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:06:27.339938 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e527f69a-d4c7-4774-a618-422af58d23fc None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:06:27.341582 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-e527f69a-d4c7-4774-a618-422af58d23fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:06:27.495225 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:06:27.506795 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e527f69a-d4c7-4774-a618-422af58d23fc None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:06:27.625312 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d6e737-1ce1-4a70-952a-3cb824e6f932 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:06:27.625834 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d6e737-1ce1-4a70-952a-3cb824e6f932 service nova] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:06:27.626111 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d6e737-1ce1-4a70-952a-3cb824e6f932 service nova] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:06:27.626398 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d6e737-1ce1-4a70-952a-3cb824e6f932 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:06:27.628032 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-93d6e737-1ce1-4a70-952a-3cb824e6f932 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:06:27.736067 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d6e737-1ce1-4a70-952a-3cb824e6f932 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:06:27.738381 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d6e737-1ce1-4a70-952a-3cb824e6f932 service nova] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:06:27.749837 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:06:27.760519 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 689/1379] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:06:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:06:30.090159 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898eec19-3e8d-4e2e-b7e4-e971cb27a198 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:06:30.091093 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898eec19-3e8d-4e2e-b7e4-e971cb27a198 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:06:30.091392 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898eec19-3e8d-4e2e-b7e4-e971cb27a198 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:06:30.092996 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 691/1380] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:06:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:06:30.098892 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e875c40-b7ee-4a96-b441-41f221366b02 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:06:30.099666 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e875c40-b7ee-4a96-b441-41f221366b02 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:06:30.099666 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e875c40-b7ee-4a96-b441-41f221366b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:06:30.182140 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-7e875c40-b7ee-4a96-b441-41f221366b02 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:06:30.183745 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-7e875c40-b7ee-4a96-b441-41f221366b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:06:30.305625 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 690/1381] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:06:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:06:30.605651 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898eec19-3e8d-4e2e-b7e4-e971cb27a198 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:06:30.607488 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-898eec19-3e8d-4e2e-b7e4-e971cb27a198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:06:30.708330 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:06:30.821780 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898eec19-3e8d-4e2e-b7e4-e971cb27a198 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:06:30.968598 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6b7879-1147-427b-b45e-dff0ffacde6d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:06:30.969201 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b7879-1147-427b-b45e-dff0ffacde6d service nova] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:06:30.969472 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b7879-1147-427b-b45e-dff0ffacde6d service nova] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:06:30.969738 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b7879-1147-427b-b45e-dff0ffacde6d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:06:30.971394 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-0c6b7879-1147-427b-b45e-dff0ffacde6d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:06:30.992675 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:06:31.080464 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6b7879-1147-427b-b45e-dff0ffacde6d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:06:31.082109 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6b7879-1147-427b-b45e-dff0ffacde6d service nova] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:06:31.102115 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 692/1382] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:06:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:06:33.354198 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c94e14-2858-4146-bbd8-287cccdc2a53 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:06:33.355064 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c94e14-2858-4146-bbd8-287cccdc2a53 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:06:33.355265 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c94e14-2858-4146-bbd8-287cccdc2a53 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:06:33.356724 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 691/1383] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:06:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 16:06:33.364684 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed2f08a-7b32-48f2-94bd-a552967e921f None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:06:33.365358 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed2f08a-7b32-48f2-94bd-a552967e921f None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:06:33.365530 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed2f08a-7b32-48f2-94bd-a552967e921f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:06:33.444661 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-9ed2f08a-7b32-48f2-94bd-a552967e921f None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:06:33.445871 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-9ed2f08a-7b32-48f2-94bd-a552967e921f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:06:33.566599 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:06:33.573613 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 693/1384] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:06:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:06:33.900295 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c94e14-2858-4146-bbd8-287cccdc2a53 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:06:33.902129 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-d4c94e14-2858-4146-bbd8-287cccdc2a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:06:34.008213 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:06:34.138235 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c94e14-2858-4146-bbd8-287cccdc2a53 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:06:34.340305 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db53cdcc-de0d-4d01-9bff-158af2a65b56 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:06:34.341106 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db53cdcc-de0d-4d01-9bff-158af2a65b56 service nova] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:06:34.341651 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db53cdcc-de0d-4d01-9bff-158af2a65b56 service nova] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:06:34.342157 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db53cdcc-de0d-4d01-9bff-158af2a65b56 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:06:34.344875 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-db53cdcc-de0d-4d01-9bff-158af2a65b56 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:06:34.398083 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:06:34.457591 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db53cdcc-de0d-4d01-9bff-158af2a65b56 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:06:34.459879 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db53cdcc-de0d-4d01-9bff-158af2a65b56 service nova] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:06:34.480596 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 692/1385] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:06:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 581 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 16:06:36.961739 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5939e4b2-07a6-4076-af35-b6f83747d19a None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:06:36.962299 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5939e4b2-07a6-4076-af35-b6f83747d19a None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:06:36.962422 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5939e4b2-07a6-4076-af35-b6f83747d19a None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:06:36.964149 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 694/1386] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:06:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:06:36.969766 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a7a538-0e80-4805-8b05-525092320a82 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:06:36.970314 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a7a538-0e80-4805-8b05-525092320a82 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:06:36.970314 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a7a538-0e80-4805-8b05-525092320a82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:06:37.042629 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-85a7a538-0e80-4805-8b05-525092320a82 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:06:37.043788 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-85a7a538-0e80-4805-8b05-525092320a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:06:37.146776 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 693/1387] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:06:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:06:37.487504 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5939e4b2-07a6-4076-af35-b6f83747d19a None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:06:37.489094 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-5939e4b2-07a6-4076-af35-b6f83747d19a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:06:37.660766 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5939e4b2-07a6-4076-af35-b6f83747d19a None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:06:37.732541 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:06:37.788882 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a2a48a-5252-486f-94f7-a3a4b126bad8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:06:37.789492 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a2a48a-5252-486f-94f7-a3a4b126bad8 service nova] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:06:37.789773 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a2a48a-5252-486f-94f7-a3a4b126bad8 service nova] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:06:37.790047 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a2a48a-5252-486f-94f7-a3a4b126bad8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:06:37.791686 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-60a2a48a-5252-486f-94f7-a3a4b126bad8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:06:38.030561 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a2a48a-5252-486f-94f7-a3a4b126bad8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:06:38.032394 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a2a48a-5252-486f-94f7-a3a4b126bad8 service nova] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:06:38.054475 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 695/1388] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:06:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 568 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 16:06:40.361476 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922fe551-2eb4-4b54-8c0d-3ce45dd7e9e0 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:06:40.362069 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922fe551-2eb4-4b54-8c0d-3ce45dd7e9e0 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:06:40.362069 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922fe551-2eb4-4b54-8c0d-3ce45dd7e9e0 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:06:40.363491 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 694/1389] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:06:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:06:40.369625 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7edd29c-5193-4eb6-b35b-f79d74b0af1e None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:06:40.370076 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7edd29c-5193-4eb6-b35b-f79d74b0af1e None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:06:40.370203 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7edd29c-5193-4eb6-b35b-f79d74b0af1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:06:40.441750 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-a7edd29c-5193-4eb6-b35b-f79d74b0af1e None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:06:40.443060 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-a7edd29c-5193-4eb6-b35b-f79d74b0af1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:06:40.479982 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:06:40.554137 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 696/1390] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:06:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:06:40.846419 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922fe551-2eb4-4b54-8c0d-3ce45dd7e9e0 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:06:40.848256 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-922fe551-2eb4-4b54-8c0d-3ce45dd7e9e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:06:40.864774 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:06:41.007184 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922fe551-2eb4-4b54-8c0d-3ce45dd7e9e0 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:06:41.104595 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:06:41.133262 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264d693e-8e7f-4271-bc50-c73787bc2838 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:06:41.133882 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264d693e-8e7f-4271-bc50-c73787bc2838 service nova] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:06:41.134148 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264d693e-8e7f-4271-bc50-c73787bc2838 service nova] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:06:41.134443 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264d693e-8e7f-4271-bc50-c73787bc2838 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:06:41.136058 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-264d693e-8e7f-4271-bc50-c73787bc2838 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:06:41.230124 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264d693e-8e7f-4271-bc50-c73787bc2838 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:06:41.231759 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264d693e-8e7f-4271-bc50-c73787bc2838 service nova] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:06:41.250254 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 695/1391] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:06:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:06:43.521662 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0799152-5dfa-4fc6-90d3-cdfdc8f2b9e0 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:06:43.522289 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0799152-5dfa-4fc6-90d3-cdfdc8f2b9e0 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:06:43.522289 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0799152-5dfa-4fc6-90d3-cdfdc8f2b9e0 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:06:43.523602 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 697/1392] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:06:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:06:43.529199 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0170f197-db5f-451d-8a52-408920ca057a None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:06:43.529636 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0170f197-db5f-451d-8a52-408920ca057a None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:06:43.529841 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0170f197-db5f-451d-8a52-408920ca057a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:06:43.601820 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-0170f197-db5f-451d-8a52-408920ca057a None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:06:43.602997 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-0170f197-db5f-451d-8a52-408920ca057a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:06:43.657195 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:06:43.705731 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 696/1393] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:06:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:06:44.000594 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0799152-5dfa-4fc6-90d3-cdfdc8f2b9e0 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:06:44.002125 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-c0799152-5dfa-4fc6-90d3-cdfdc8f2b9e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:06:44.155786 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0799152-5dfa-4fc6-90d3-cdfdc8f2b9e0 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:06:44.173786 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:06:44.292876 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faca326-1f8c-43b0-9e97-0553e76f59c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:06:44.293602 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faca326-1f8c-43b0-9e97-0553e76f59c4 service nova] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:06:44.293885 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faca326-1f8c-43b0-9e97-0553e76f59c4 service nova] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:06:44.294153 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faca326-1f8c-43b0-9e97-0553e76f59c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:06:44.296046 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-0faca326-1f8c-43b0-9e97-0553e76f59c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:06:44.402447 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faca326-1f8c-43b0-9e97-0553e76f59c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:06:44.404054 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faca326-1f8c-43b0-9e97-0553e76f59c4 service nova] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:06:44.423115 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 698/1394] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:06:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:06:46.272403 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ed0891-34c3-4f6e-9923-8f5144279f94 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:06:46.273300 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ed0891-34c3-4f6e-9923-8f5144279f94 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:06:46.273300 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ed0891-34c3-4f6e-9923-8f5144279f94 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:06:46.274479 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 697/1395] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:06:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:06:46.279906 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bbd3b3-abf2-4ab6-b60b-3a0ee9aacf30 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:06:46.280365 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bbd3b3-abf2-4ab6-b60b-3a0ee9aacf30 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:06:46.280593 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bbd3b3-abf2-4ab6-b60b-3a0ee9aacf30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:06:46.308010 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:06:46.352392 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-94bbd3b3-abf2-4ab6-b60b-3a0ee9aacf30 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:06:46.353593 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-94bbd3b3-abf2-4ab6-b60b-3a0ee9aacf30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:06:46.454760 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 699/1396] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:06:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:06:46.754220 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6695484-1153-4b59-a8a9-3b2f491008fd None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:06:46.754550 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6695484-1153-4b59-a8a9-3b2f491008fd None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:06:46.754830 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6695484-1153-4b59-a8a9-3b2f491008fd None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:06:46.756202 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 698/1397] 10.222.0.57 () {54 vars in 962 bytes} [Mon Sep 19 16:06:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:06:46.764493 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b193877f-6e19-4177-901f-0c203e8c0713 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:06:46.764837 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b193877f-6e19-4177-901f-0c203e8c0713 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:06:46.765269 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b193877f-6e19-4177-901f-0c203e8c0713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:06:46.850726 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-b193877f-6e19-4177-901f-0c203e8c0713 None None] MFA Rules not processed for user `5632ade0cc6f4ae28f72160de68486cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:06:46.852054 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-b193877f-6e19-4177-901f-0c203e8c0713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:06:46.873378 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f843d4-c338-48e9-9a77-074cfbcd44af None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:06:46.873716 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f843d4-c338-48e9-9a77-074cfbcd44af None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:06:46.874053 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f843d4-c338-48e9-9a77-074cfbcd44af None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:06:46.875929 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 699/1398] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:06:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 16:06:46.882388 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca78dc04-95a9-4b98-ada1-acc471d17e6f None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:06:46.882682 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca78dc04-95a9-4b98-ada1-acc471d17e6f None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:06:46.882963 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca78dc04-95a9-4b98-ada1-acc471d17e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:06:46.901677 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:06:46.942436 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 700/1399] 10.222.0.57 () {58 vars in 1059 bytes} [Mon Sep 19 16:06:46 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:06:46.951661 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b193877f-6e19-4177-901f-0c203e8c0713 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:06:46.957260 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-ca78dc04-95a9-4b98-ada1-acc471d17e6f None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:06:46.958883 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-ca78dc04-95a9-4b98-ada1-acc471d17e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:06:46.959899 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b193877f-6e19-4177-901f-0c203e8c0713 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:06:47.067033 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 700/1400] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:06:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:06:47.075898 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e90bc7-9937-413d-8ad8-87ee40ea68f8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5632ade0cc6f4ae28f72160de68486cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:06:47.076939 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e90bc7-9937-413d-8ad8-87ee40ea68f8 service glance] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:06:47.077428 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e90bc7-9937-413d-8ad8-87ee40ea68f8 service glance] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:06:47.077913 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e90bc7-9937-413d-8ad8-87ee40ea68f8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:06:47.081506 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-17e90bc7-9937-413d-8ad8-87ee40ea68f8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:06:47.203420 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e90bc7-9937-413d-8ad8-87ee40ea68f8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:06:47.204855 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e90bc7-9937-413d-8ad8-87ee40ea68f8 service glance] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:06:47.210510 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:06:47.232381 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 701/1401] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:06:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 16:06:47.357268 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca78dc04-95a9-4b98-ada1-acc471d17e6f None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:06:47.358503 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-ca78dc04-95a9-4b98-ada1-acc471d17e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:06:47.398563 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:06:47.492389 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca78dc04-95a9-4b98-ada1-acc471d17e6f None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:06:47.616164 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5792001-8b94-4439-b235-3e66b91a321f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:06:47.616861 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5792001-8b94-4439-b235-3e66b91a321f service nova] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:06:47.616861 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5792001-8b94-4439-b235-3e66b91a321f service nova] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:06:47.617134 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5792001-8b94-4439-b235-3e66b91a321f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:06:47.618899 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-f5792001-8b94-4439-b235-3e66b91a321f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:06:47.655113 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:06:47.724050 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5792001-8b94-4439-b235-3e66b91a321f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:06:47.725640 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5792001-8b94-4439-b235-3e66b91a321f service nova] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:06:47.744541 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 701/1402] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:06:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:06:48.046643 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03590ae7-263a-4ce9-af08-26edec07b03c None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:06:48.047178 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03590ae7-263a-4ce9-af08-26edec07b03c None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:06:48.047440 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03590ae7-263a-4ce9-af08-26edec07b03c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:06:48.118690 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-03590ae7-263a-4ce9-af08-26edec07b03c None None] MFA Rules not processed for user `7222bd3b313b47e5a4fa0ebc9ff0cdd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:06:48.119921 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-03590ae7-263a-4ce9-af08-26edec07b03c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:06:48.222126 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 702/1403] 10.222.0.57 () {58 vars in 1009 bytes} [Mon Sep 19 16:06:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:06:48.264589 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0d6ff-f9f7-4114-a1c4-f9860a7abc67 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:06:48.264870 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0d6ff-f9f7-4114-a1c4-f9860a7abc67 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:06:48.265140 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0d6ff-f9f7-4114-a1c4-f9860a7abc67 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:06:48.266522 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 702/1404] 10.222.0.57 () {54 vars in 943 bytes} [Mon Sep 19 16:06:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:06:48.273788 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84eff26-3bd5-4036-a2b7-85b1c51126f7 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:06:48.274124 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84eff26-3bd5-4036-a2b7-85b1c51126f7 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:06:48.274456 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84eff26-3bd5-4036-a2b7-85b1c51126f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:06:48.365341 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-e84eff26-3bd5-4036-a2b7-85b1c51126f7 None None] MFA Rules not processed for user `884a0524aca744b99c7496c11c3291ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:06:48.366611 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-e84eff26-3bd5-4036-a2b7-85b1c51126f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:06:48.389561 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:06:48.477902 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 703/1405] 10.222.0.57 () {58 vars in 1040 bytes} [Mon Sep 19 16:06:48 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:06:48.488754 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e0d6ff-f9f7-4114-a1c4-f9860a7abc67 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:06:48.490292 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-90e0d6ff-f9f7-4114-a1c4-f9860a7abc67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:06:48.644932 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e0d6ff-f9f7-4114-a1c4-f9860a7abc67 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:06:48.664917 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:06:48.769057 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33304101-4bb1-4394-9e55-1345c2622648 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '884a0524aca744b99c7496c11c3291ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:06:48.769625 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33304101-4bb1-4394-9e55-1345c2622648 service swift] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:06:48.769896 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33304101-4bb1-4394-9e55-1345c2622648 service swift] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:06:48.770167 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33304101-4bb1-4394-9e55-1345c2622648 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:06:48.771871 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-33304101-4bb1-4394-9e55-1345c2622648 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:06:48.865366 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33304101-4bb1-4394-9e55-1345c2622648 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:06:48.867007 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33304101-4bb1-4394-9e55-1345c2622648 service swift] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:06:48.875037 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 703/1406] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Sep 19 16:06:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 19 16:06:50.142944 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755b1e91-073a-4a69-8326-3ded8d107295 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:06:50.143979 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755b1e91-073a-4a69-8326-3ded8d107295 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:06:50.143979 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755b1e91-073a-4a69-8326-3ded8d107295 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:06:50.145494 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 704/1407] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:06:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:06:50.151160 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa4cfa-8c11-4fbf-bc0d-9df0c788c94d None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:06:50.151643 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa4cfa-8c11-4fbf-bc0d-9df0c788c94d None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:06:50.151733 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa4cfa-8c11-4fbf-bc0d-9df0c788c94d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:06:50.227850 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:06:50.232342 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-a2aa4cfa-8c11-4fbf-bc0d-9df0c788c94d None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:06:50.234118 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-a2aa4cfa-8c11-4fbf-bc0d-9df0c788c94d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:06:50.348961 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 704/1408] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:06:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:06:50.647065 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755b1e91-073a-4a69-8326-3ded8d107295 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:06:50.649294 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-755b1e91-073a-4a69-8326-3ded8d107295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:06:50.833123 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:06:50.842201 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755b1e91-073a-4a69-8326-3ded8d107295 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:06:50.989056 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5764d2e0-c29f-4ef0-9f80-b7264a5f5557 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:06:50.989056 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5764d2e0-c29f-4ef0-9f80-b7264a5f5557 service nova] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:06:50.989056 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5764d2e0-c29f-4ef0-9f80-b7264a5f5557 service nova] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:06:50.989056 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5764d2e0-c29f-4ef0-9f80-b7264a5f5557 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:06:50.989649 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-5764d2e0-c29f-4ef0-9f80-b7264a5f5557 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:06:51.095272 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5764d2e0-c29f-4ef0-9f80-b7264a5f5557 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:06:51.097199 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5764d2e0-c29f-4ef0-9f80-b7264a5f5557 service nova] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:06:51.113682 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:06:51.117438 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 705/1409] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:06:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:07:00.345432 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=43 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:07:12.194440 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e257b8bb-1f85-4b75-ba0f-c16b6099b214 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:07:12.195082 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e257b8bb-1f85-4b75-ba0f-c16b6099b214 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:07:12.195082 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e257b8bb-1f85-4b75-ba0f-c16b6099b214 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:07:12.196218 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 705/1410] 10.222.0.57 () {54 vars in 918 bytes} [Mon Sep 19 16:07:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:07:12.201283 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf6b983-16f5-4ed9-97f8-f25a60746528 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:07:12.201751 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf6b983-16f5-4ed9-97f8-f25a60746528 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:07:12.201859 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf6b983-16f5-4ed9-97f8-f25a60746528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:07:12.280276 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-ccf6b983-16f5-4ed9-97f8-f25a60746528 None None] MFA Rules not processed for user `1f1a985495ed48a4acc48e2e6fc7f977`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:07:12.281512 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-ccf6b983-16f5-4ed9-97f8-f25a60746528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:07:12.398381 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 706/1411] 10.222.0.57 () {58 vars in 1015 bytes} [Mon Sep 19 16:07:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:07:12.413412 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66e99fe-752b-4080-8291-fff9ed96dddb None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:07:12.413768 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66e99fe-752b-4080-8291-fff9ed96dddb None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:07:12.414070 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66e99fe-752b-4080-8291-fff9ed96dddb None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:07:12.415465 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 706/1412] 10.222.0.57 () {54 vars in 918 bytes} [Mon Sep 19 16:07:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:07:12.420762 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4debfe7-8d51-41cc-9042-7f152cc42096 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:07:12.421332 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4debfe7-8d51-41cc-9042-7f152cc42096 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:07:12.421609 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4debfe7-8d51-41cc-9042-7f152cc42096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:07:12.499485 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:07:12.503153 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-c4debfe7-8d51-41cc-9042-7f152cc42096 None None] MFA Rules not processed for user `1ecd01cc72834b72b5ff2c4b47249737`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:07:12.505104 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-c4debfe7-8d51-41cc-9042-7f152cc42096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:07:12.624439 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 707/1413] 10.222.0.57 () {58 vars in 1015 bytes} [Mon Sep 19 16:07:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:07:15.269101 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd26fea5-23d4-4c98-b653-ab3bdf46b4ec None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:07:15.269693 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd26fea5-23d4-4c98-b653-ab3bdf46b4ec None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:07:15.269693 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd26fea5-23d4-4c98-b653-ab3bdf46b4ec None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:07:15.271018 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 707/1414] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 16:07:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:07:15.276382 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ada6cf-3e96-47c4-94bc-bd3a2e4ae8ca None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:07:15.276800 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ada6cf-3e96-47c4-94bc-bd3a2e4ae8ca None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:07:15.276914 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ada6cf-3e96-47c4-94bc-bd3a2e4ae8ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:07:15.348828 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-97ada6cf-3e96-47c4-94bc-bd3a2e4ae8ca None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:07:15.350076 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-97ada6cf-3e96-47c4-94bc-bd3a2e4ae8ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:07:15.419669 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:07:15.461860 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 708/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 16:07:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:07:15.754409 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd26fea5-23d4-4c98-b653-ab3bdf46b4ec None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:07:15.755986 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-fd26fea5-23d4-4c98-b653-ab3bdf46b4ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:07:15.905319 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd26fea5-23d4-4c98-b653-ab3bdf46b4ec None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:07:15.978436 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:07:16.020705 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1934b03d-728b-4d26-ab71-fdb7ef635c22 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:07:16.021228 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1934b03d-728b-4d26-ab71-fdb7ef635c22 service nova] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:07:16.021512 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1934b03d-728b-4d26-ab71-fdb7ef635c22 service nova] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:07:16.021783 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1934b03d-728b-4d26-ab71-fdb7ef635c22 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:07:16.023478 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-1934b03d-728b-4d26-ab71-fdb7ef635c22 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:07:16.228715 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1934b03d-728b-4d26-ab71-fdb7ef635c22 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:07:16.230300 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1934b03d-728b-4d26-ab71-fdb7ef635c22 service nova] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:07:16.249035 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 708/1416] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:07:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:07:17.401094 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97601dc0-1061-44f6-9fe9-20d315dd5891 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:07:17.401666 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97601dc0-1061-44f6-9fe9-20d315dd5891 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:07:17.401666 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97601dc0-1061-44f6-9fe9-20d315dd5891 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:07:17.403028 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 709/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 16:07:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:07:17.409342 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1827bc5-a5bf-4999-856d-749ca6f28d23 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:07:17.409839 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1827bc5-a5bf-4999-856d-749ca6f28d23 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:07:17.409949 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1827bc5-a5bf-4999-856d-749ca6f28d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:07:17.486129 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-c1827bc5-a5bf-4999-856d-749ca6f28d23 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:07:17.487405 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-c1827bc5-a5bf-4999-856d-749ca6f28d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:07:17.514885 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:07:17.584616 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 709/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 16:07:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:07:17.735415 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97601dc0-1061-44f6-9fe9-20d315dd5891 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:07:17.736968 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-97601dc0-1061-44f6-9fe9-20d315dd5891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:07:17.888971 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97601dc0-1061-44f6-9fe9-20d315dd5891 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:07:17.929544 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:07:18.002485 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa45e818-5ea6-4e62-a1d2-7f1f76fa7070 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:07:18.003013 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa45e818-5ea6-4e62-a1d2-7f1f76fa7070 service nova] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:07:18.003295 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa45e818-5ea6-4e62-a1d2-7f1f76fa7070 service nova] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:07:18.003567 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa45e818-5ea6-4e62-a1d2-7f1f76fa7070 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:07:18.005191 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-fa45e818-5ea6-4e62-a1d2-7f1f76fa7070 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:07:18.097134 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa45e818-5ea6-4e62-a1d2-7f1f76fa7070 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:07:18.098759 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa45e818-5ea6-4e62-a1d2-7f1f76fa7070 service nova] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:07:18.116501 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 710/1419] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:07:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:07:19.265469 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27b12b7-fc20-4da3-bc69-719a3de2b438 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:07:19.266055 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27b12b7-fc20-4da3-bc69-719a3de2b438 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:07:19.266055 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27b12b7-fc20-4da3-bc69-719a3de2b438 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:07:19.267387 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 710/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 16:07:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 16:07:19.273773 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8bdee5-4ee9-46b8-abff-8ec5b3373793 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:07:19.274231 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8bdee5-4ee9-46b8-abff-8ec5b3373793 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:07:19.274358 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8bdee5-4ee9-46b8-abff-8ec5b3373793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:07:19.329266 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:07:19.351712 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-5c8bdee5-4ee9-46b8-abff-8ec5b3373793 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:07:19.352933 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-5c8bdee5-4ee9-46b8-abff-8ec5b3373793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:07:19.450058 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 711/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 16:07:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:07:19.601700 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27b12b7-fc20-4da3-bc69-719a3de2b438 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:07:19.603288 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-c27b12b7-fc20-4da3-bc69-719a3de2b438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:07:19.752663 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27b12b7-fc20-4da3-bc69-719a3de2b438 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:07:19.765244 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:07:19.872064 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ffa7b4-aa3d-41c2-837b-03b13172ed95 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:07:19.872548 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ffa7b4-aa3d-41c2-837b-03b13172ed95 service nova] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:07:19.872818 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ffa7b4-aa3d-41c2-837b-03b13172ed95 service nova] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:07:19.873085 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ffa7b4-aa3d-41c2-837b-03b13172ed95 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:07:19.874694 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-f0ffa7b4-aa3d-41c2-837b-03b13172ed95 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:07:19.966288 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ffa7b4-aa3d-41c2-837b-03b13172ed95 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:07:19.967956 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ffa7b4-aa3d-41c2-837b-03b13172ed95 service nova] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:07:19.986422 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 711/1422] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:07:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:07:21.153079 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef3ffc2-3f9e-4a34-86eb-e21a604b4b5e None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:07:21.153598 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef3ffc2-3f9e-4a34-86eb-e21a604b4b5e None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:07:21.153598 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef3ffc2-3f9e-4a34-86eb-e21a604b4b5e None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:07:21.155007 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 712/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 16:07:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:07:21.162033 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512321c7-1d1f-4a37-be9f-87df4ffbed2f None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:07:21.162625 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512321c7-1d1f-4a37-be9f-87df4ffbed2f None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:07:21.162625 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512321c7-1d1f-4a37-be9f-87df4ffbed2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:07:21.183202 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:07:21.243752 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-512321c7-1d1f-4a37-be9f-87df4ffbed2f None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:07:21.244999 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-512321c7-1d1f-4a37-be9f-87df4ffbed2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:07:21.352942 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 712/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 16:07:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:07:21.508859 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef3ffc2-3f9e-4a34-86eb-e21a604b4b5e None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:07:21.510442 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-9ef3ffc2-3f9e-4a34-86eb-e21a604b4b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:07:21.620716 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:07:21.659240 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef3ffc2-3f9e-4a34-86eb-e21a604b4b5e None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:07:21.772705 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0badf831-1ab9-4997-a0b6-861aba9073ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:07:21.773242 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0badf831-1ab9-4997-a0b6-861aba9073ce service nova] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:07:21.773518 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0badf831-1ab9-4997-a0b6-861aba9073ce service nova] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:07:21.773786 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0badf831-1ab9-4997-a0b6-861aba9073ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:07:21.775435 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-0badf831-1ab9-4997-a0b6-861aba9073ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:07:21.851444 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:07:21.865495 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0badf831-1ab9-4997-a0b6-861aba9073ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:07:21.867119 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0badf831-1ab9-4997-a0b6-861aba9073ce service nova] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:07:21.885231 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 713/1425] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:07:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:07:23.040857 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb3687d-8c52-4b86-b585-5e2f38770d5a None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:07:23.041722 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb3687d-8c52-4b86-b585-5e2f38770d5a None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:07:23.041722 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb3687d-8c52-4b86-b585-5e2f38770d5a None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:07:23.042577 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 713/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 16:07:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:07:23.049966 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c913432-7a36-4de8-bd99-99c6cc60935a None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:07:23.050473 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c913432-7a36-4de8-bd99-99c6cc60935a None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:07:23.050585 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c913432-7a36-4de8-bd99-99c6cc60935a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:07:23.126714 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-5c913432-7a36-4de8-bd99-99c6cc60935a None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:07:23.128549 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-5c913432-7a36-4de8-bd99-99c6cc60935a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:07:23.226059 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 714/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 16:07:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:07:23.377252 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb3687d-8c52-4b86-b585-5e2f38770d5a None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:07:23.378856 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-0cb3687d-8c52-4b86-b585-5e2f38770d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:07:23.468691 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:07:23.538343 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb3687d-8c52-4b86-b585-5e2f38770d5a None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:07:23.659732 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c11a0ca-ab4f-40f9-a20b-60b98d3b4ed4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:07:23.660268 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c11a0ca-ab4f-40f9-a20b-60b98d3b4ed4 service nova] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:07:23.660545 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c11a0ca-ab4f-40f9-a20b-60b98d3b4ed4 service nova] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:07:23.660817 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c11a0ca-ab4f-40f9-a20b-60b98d3b4ed4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:07:23.662549 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-7c11a0ca-ab4f-40f9-a20b-60b98d3b4ed4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:07:23.717236 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:07:23.763765 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c11a0ca-ab4f-40f9-a20b-60b98d3b4ed4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:07:23.765391 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c11a0ca-ab4f-40f9-a20b-60b98d3b4ed4 service nova] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:07:23.789204 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 714/1428] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:07:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:07:24.872424 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ba033-d9f7-4cab-9ed2-d0e9b874074d None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:07:24.873065 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ba033-d9f7-4cab-9ed2-d0e9b874074d None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:07:24.873065 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ba033-d9f7-4cab-9ed2-d0e9b874074d None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:07:24.874290 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 715/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 16:07:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:07:24.881252 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41effe-fff1-45a2-b69d-f3cf2ea0dd5c None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:07:24.881769 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41effe-fff1-45a2-b69d-f3cf2ea0dd5c None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:07:24.882007 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41effe-fff1-45a2-b69d-f3cf2ea0dd5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:07:24.962225 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-ad41effe-fff1-45a2-b69d-f3cf2ea0dd5c None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:07:24.963473 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-ad41effe-fff1-45a2-b69d-f3cf2ea0dd5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:07:25.062195 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 715/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 16:07:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:07:25.223522 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96ba033-d9f7-4cab-9ed2-d0e9b874074d None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:07:25.225057 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-b96ba033-d9f7-4cab-9ed2-d0e9b874074d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:07:25.275410 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:07:25.375506 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96ba033-d9f7-4cab-9ed2-d0e9b874074d None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:07:25.488565 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde34d3f-ac70-4bb4-ba96-06aa99d99c99 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:07:25.489173 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde34d3f-ac70-4bb4-ba96-06aa99d99c99 service nova] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:07:25.489448 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde34d3f-ac70-4bb4-ba96-06aa99d99c99 service nova] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:07:25.489719 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde34d3f-ac70-4bb4-ba96-06aa99d99c99 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:07:25.491378 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-dde34d3f-ac70-4bb4-ba96-06aa99d99c99 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:07:25.498542 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:07:25.584256 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde34d3f-ac70-4bb4-ba96-06aa99d99c99 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:07:25.585862 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde34d3f-ac70-4bb4-ba96-06aa99d99c99 service nova] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:07:25.606073 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 716/1431] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:07:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:07:26.684297 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fb60a4-6f15-47ab-bda9-90a5df184df7 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:07:26.684878 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fb60a4-6f15-47ab-bda9-90a5df184df7 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:07:26.684878 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fb60a4-6f15-47ab-bda9-90a5df184df7 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:07:26.686174 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 716/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 16:07:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:07:26.693680 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e512274-1700-4435-b571-24cdae351cd9 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:07:26.694201 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e512274-1700-4435-b571-24cdae351cd9 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:07:26.694201 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e512274-1700-4435-b571-24cdae351cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:07:26.778289 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-2e512274-1700-4435-b571-24cdae351cd9 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:07:26.779547 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-2e512274-1700-4435-b571-24cdae351cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:07:26.851119 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:07:26.884923 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 717/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 16:07:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:07:27.035606 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fb60a4-6f15-47ab-bda9-90a5df184df7 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:07:27.037176 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-70fb60a4-6f15-47ab-bda9-90a5df184df7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:07:27.055837 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:07:27.194172 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fb60a4-6f15-47ab-bda9-90a5df184df7 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:07:27.301629 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:07:27.332462 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e2ab11-032d-49c7-8f68-02f698a553ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:07:27.333214 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e2ab11-032d-49c7-8f68-02f698a553ef service nova] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:07:27.333493 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e2ab11-032d-49c7-8f68-02f698a553ef service nova] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:07:27.333768 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e2ab11-032d-49c7-8f68-02f698a553ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:07:27.335423 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-f3e2ab11-032d-49c7-8f68-02f698a553ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:07:27.435366 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e2ab11-032d-49c7-8f68-02f698a553ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:07:27.436989 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e2ab11-032d-49c7-8f68-02f698a553ef service nova] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:07:27.456748 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 717/1434] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:07:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 16:07:28.553439 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3748bbb1-6479-43b4-ab1b-f203c64db3a3 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:07:28.554016 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3748bbb1-6479-43b4-ab1b-f203c64db3a3 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:07:28.554016 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3748bbb1-6479-43b4-ab1b-f203c64db3a3 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:07:28.555369 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 718/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 16:07:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:07:28.562048 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cde5b5-327b-4e52-a4d3-5b5570e84433 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:07:28.562588 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cde5b5-327b-4e52-a4d3-5b5570e84433 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:07:28.562686 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cde5b5-327b-4e52-a4d3-5b5570e84433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:07:28.641535 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-e1cde5b5-327b-4e52-a4d3-5b5570e84433 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:07:28.643370 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-e1cde5b5-327b-4e52-a4d3-5b5570e84433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:07:28.685228 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:07:28.741052 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 718/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 16:07:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:07:28.896280 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3748bbb1-6479-43b4-ab1b-f203c64db3a3 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:07:28.897790 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-3748bbb1-6479-43b4-ab1b-f203c64db3a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:07:29.055472 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3748bbb1-6479-43b4-ab1b-f203c64db3a3 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:07:29.111802 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:07:29.174494 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ec066a-8233-4914-8f41-62358d8179ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:07:29.175030 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ec066a-8233-4914-8f41-62358d8179ba service nova] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:07:29.175311 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ec066a-8233-4914-8f41-62358d8179ba service nova] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:07:29.175584 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ec066a-8233-4914-8f41-62358d8179ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:07:29.177207 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-38ec066a-8233-4914-8f41-62358d8179ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:07:29.276755 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ec066a-8233-4914-8f41-62358d8179ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:07:29.278364 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ec066a-8233-4914-8f41-62358d8179ba service nova] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:07:29.299993 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 719/1437] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:07:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:07:30.375659 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70898ad0-1801-418b-9ecf-4cdecf8101eb None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:07:30.376208 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70898ad0-1801-418b-9ecf-4cdecf8101eb None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:07:30.376208 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70898ad0-1801-418b-9ecf-4cdecf8101eb None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:07:30.378236 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 719/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 16:07:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:07:30.385010 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7729476-ca46-4e8f-91bd-da39203ac468 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:07:30.385472 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7729476-ca46-4e8f-91bd-da39203ac468 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:07:30.385579 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7729476-ca46-4e8f-91bd-da39203ac468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:07:30.463770 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-d7729476-ca46-4e8f-91bd-da39203ac468 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:07:30.465012 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-d7729476-ca46-4e8f-91bd-da39203ac468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:07:30.467131 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:07:30.568723 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 720/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 16:07:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:07:30.729783 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70898ad0-1801-418b-9ecf-4cdecf8101eb None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:07:30.731392 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-70898ad0-1801-418b-9ecf-4cdecf8101eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:07:30.892944 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70898ad0-1801-418b-9ecf-4cdecf8101eb None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:07:30.921498 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:07:31.011326 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bdbcc0-6c45-4933-9bd3-0240f3d98b7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:07:31.011836 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bdbcc0-6c45-4933-9bd3-0240f3d98b7a service nova] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:07:31.012107 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bdbcc0-6c45-4933-9bd3-0240f3d98b7a service nova] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:07:31.012372 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bdbcc0-6c45-4933-9bd3-0240f3d98b7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:07:31.014007 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-30bdbcc0-6c45-4933-9bd3-0240f3d98b7a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:07:31.105717 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bdbcc0-6c45-4933-9bd3-0240f3d98b7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:07:31.107345 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bdbcc0-6c45-4933-9bd3-0240f3d98b7a service nova] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:07:31.125233 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 720/1440] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:07:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:07:33.235632 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e8dfe-e4c2-4bbc-9d05-c128caaa4c41 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:07:33.236304 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e8dfe-e4c2-4bbc-9d05-c128caaa4c41 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:07:33.236304 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e8dfe-e4c2-4bbc-9d05-c128caaa4c41 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:07:33.237626 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 721/1441] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 16:07:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:07:33.243094 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e90649-2541-4b58-8c01-05d20e60bb36 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:07:33.243544 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e90649-2541-4b58-8c01-05d20e60bb36 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:07:33.243668 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e90649-2541-4b58-8c01-05d20e60bb36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:07:33.285675 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:07:33.311346 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-f3e90649-2541-4b58-8c01-05d20e60bb36 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:07:33.312571 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-f3e90649-2541-4b58-8c01-05d20e60bb36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:07:33.419742 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 721/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 16:07:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:07:33.702396 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257e8dfe-e4c2-4bbc-9d05-c128caaa4c41 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:07:33.703956 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-257e8dfe-e4c2-4bbc-9d05-c128caaa4c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:07:33.809361 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257e8dfe-e4c2-4bbc-9d05-c128caaa4c41 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:07:33.843111 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:07:33.888093 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f0aa58-f86e-4fbf-b6d8-59ad38b26f54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:07:33.888702 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f0aa58-f86e-4fbf-b6d8-59ad38b26f54 service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:07:33.888982 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f0aa58-f86e-4fbf-b6d8-59ad38b26f54 service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:07:33.889256 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f0aa58-f86e-4fbf-b6d8-59ad38b26f54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:07:33.890900 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-c1f0aa58-f86e-4fbf-b6d8-59ad38b26f54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:07:34.000649 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f0aa58-f86e-4fbf-b6d8-59ad38b26f54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:07:34.002664 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f0aa58-f86e-4fbf-b6d8-59ad38b26f54 service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:07:34.026927 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 722/1443] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:07:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:07:36.368635 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d663f193-3032-4e96-ab0b-4a92488d7b55 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:07:36.369212 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d663f193-3032-4e96-ab0b-4a92488d7b55 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:07:36.369212 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d663f193-3032-4e96-ab0b-4a92488d7b55 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:07:36.370580 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 722/1444] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 16:07:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:07:36.376066 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03123fb6-f306-4a32-a7a8-042e9493d5c5 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:07:36.376548 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03123fb6-f306-4a32-a7a8-042e9493d5c5 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:07:36.376652 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03123fb6-f306-4a32-a7a8-042e9493d5c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:07:36.444667 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-03123fb6-f306-4a32-a7a8-042e9493d5c5 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:07:36.445891 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-03123fb6-f306-4a32-a7a8-042e9493d5c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:07:36.459295 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:07:36.543331 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 723/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 16:07:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:07:36.834806 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d663f193-3032-4e96-ab0b-4a92488d7b55 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:07:36.836380 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-d663f193-3032-4e96-ab0b-4a92488d7b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:07:36.943149 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d663f193-3032-4e96-ab0b-4a92488d7b55 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:07:36.956774 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:07:37.023314 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14057d6d-fda7-4e0b-90ce-040955b001e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:07:37.023835 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14057d6d-fda7-4e0b-90ce-040955b001e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:07:37.024143 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14057d6d-fda7-4e0b-90ce-040955b001e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:07:37.024379 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14057d6d-fda7-4e0b-90ce-040955b001e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:07:37.025997 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-14057d6d-fda7-4e0b-90ce-040955b001e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:07:37.136386 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14057d6d-fda7-4e0b-90ce-040955b001e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:07:37.137836 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14057d6d-fda7-4e0b-90ce-040955b001e9 service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:07:37.156544 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 723/1446] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:07:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:07:39.760928 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b49bf-168f-455f-8637-e6c671e97aa1 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:07:39.761510 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b49bf-168f-455f-8637-e6c671e97aa1 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:07:39.761611 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b49bf-168f-455f-8637-e6c671e97aa1 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:07:39.763137 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 724/1447] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 16:07:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:07:39.769264 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44535bf7-33b3-44ae-8714-4f1eed528f24 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:07:39.769787 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44535bf7-33b3-44ae-8714-4f1eed528f24 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:07:39.769787 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44535bf7-33b3-44ae-8714-4f1eed528f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:07:39.820374 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:07:39.838693 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-44535bf7-33b3-44ae-8714-4f1eed528f24 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:07:39.839912 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-44535bf7-33b3-44ae-8714-4f1eed528f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:07:39.940120 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 724/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 16:07:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:07:40.226239 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4b49bf-168f-455f-8637-e6c671e97aa1 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:07:40.227811 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-da4b49bf-168f-455f-8637-e6c671e97aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:07:40.331886 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4b49bf-168f-455f-8637-e6c671e97aa1 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:07:40.373913 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:07:40.408442 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838b6b86-84d0-4cb5-a48f-1dac79a7eb27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:07:40.409042 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838b6b86-84d0-4cb5-a48f-1dac79a7eb27 service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:07:40.409400 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838b6b86-84d0-4cb5-a48f-1dac79a7eb27 service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:07:40.409593 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838b6b86-84d0-4cb5-a48f-1dac79a7eb27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:07:40.411220 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-838b6b86-84d0-4cb5-a48f-1dac79a7eb27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:07:40.528186 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838b6b86-84d0-4cb5-a48f-1dac79a7eb27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:07:40.529546 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838b6b86-84d0-4cb5-a48f-1dac79a7eb27 service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:07:40.548000 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 725/1449] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:07:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:07:42.918973 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab542b7b-d343-4794-bb45-74be15b8016d None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:07:42.919535 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab542b7b-d343-4794-bb45-74be15b8016d None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:07:42.919535 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab542b7b-d343-4794-bb45-74be15b8016d None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:07:42.920872 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 725/1450] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 16:07:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:07:42.926872 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72faf76b-d12b-46db-a467-5c00235fe5d2 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:07:42.927345 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72faf76b-d12b-46db-a467-5c00235fe5d2 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:07:42.927479 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72faf76b-d12b-46db-a467-5c00235fe5d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:07:43.014600 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-72faf76b-d12b-46db-a467-5c00235fe5d2 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:07:43.015860 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-72faf76b-d12b-46db-a467-5c00235fe5d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:07:43.040003 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:07:43.135912 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 726/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 16:07:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:07:43.420373 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab542b7b-d343-4794-bb45-74be15b8016d None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:07:43.422149 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-ab542b7b-d343-4794-bb45-74be15b8016d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:07:43.535451 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab542b7b-d343-4794-bb45-74be15b8016d None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:07:43.556010 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:07:43.609418 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1aae65-88ab-41dd-91df-1c672b63e9eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:07:43.610021 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1aae65-88ab-41dd-91df-1c672b63e9eb service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:07:43.610300 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1aae65-88ab-41dd-91df-1c672b63e9eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:07:43.610616 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1aae65-88ab-41dd-91df-1c672b63e9eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:07:43.612318 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-7c1aae65-88ab-41dd-91df-1c672b63e9eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:07:43.722056 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1aae65-88ab-41dd-91df-1c672b63e9eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:07:43.723519 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1aae65-88ab-41dd-91df-1c672b63e9eb service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:07:43.741643 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 726/1452] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:07:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:07:45.971432 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92907e96-7c5d-4137-9385-bb233571dbd4 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:07:45.972015 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92907e96-7c5d-4137-9385-bb233571dbd4 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:07:45.972015 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92907e96-7c5d-4137-9385-bb233571dbd4 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:07:45.974053 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 727/1453] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 16:07:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 16:07:45.979628 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c86f8b-d398-4cd3-a51b-d4029f6d64cb None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:07:45.980156 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c86f8b-d398-4cd3-a51b-d4029f6d64cb None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:07:45.980156 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c86f8b-d398-4cd3-a51b-d4029f6d64cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:07:46.043408 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:07:46.046821 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-22c86f8b-d398-4cd3-a51b-d4029f6d64cb None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:07:46.048714 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-22c86f8b-d398-4cd3-a51b-d4029f6d64cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:07:46.152077 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 727/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 16:07:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:07:46.437120 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92907e96-7c5d-4137-9385-bb233571dbd4 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:07:46.438886 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-92907e96-7c5d-4137-9385-bb233571dbd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:07:46.543545 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92907e96-7c5d-4137-9385-bb233571dbd4 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:07:46.594097 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:07:46.619706 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cefe840-0cb0-4a89-903b-36cb7b72bbbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:07:46.620310 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cefe840-0cb0-4a89-903b-36cb7b72bbbc service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:07:46.620583 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cefe840-0cb0-4a89-903b-36cb7b72bbbc service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:07:46.620848 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cefe840-0cb0-4a89-903b-36cb7b72bbbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:07:46.622474 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-6cefe840-0cb0-4a89-903b-36cb7b72bbbc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:07:46.734688 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cefe840-0cb0-4a89-903b-36cb7b72bbbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:07:46.736821 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cefe840-0cb0-4a89-903b-36cb7b72bbbc service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:07:46.754604 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 728/1455] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:07:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:07:48.983546 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b3a3bf-adfa-42d6-9790-5e7c3337b334 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:07:48.984075 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b3a3bf-adfa-42d6-9790-5e7c3337b334 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:07:48.984075 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b3a3bf-adfa-42d6-9790-5e7c3337b334 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:07:48.985432 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 728/1456] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 16:07:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:07:48.990873 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd16a79-b03d-49b7-9308-336b85626de1 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:07:48.991330 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd16a79-b03d-49b7-9308-336b85626de1 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:07:48.991436 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd16a79-b03d-49b7-9308-336b85626de1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:07:49.064452 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-cfd16a79-b03d-49b7-9308-336b85626de1 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:07:49.065684 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-cfd16a79-b03d-49b7-9308-336b85626de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:07:49.097134 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:07:49.168215 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 729/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 16:07:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:07:49.457686 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b3a3bf-adfa-42d6-9790-5e7c3337b334 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:07:49.459266 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-17b3a3bf-adfa-42d6-9790-5e7c3337b334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:07:49.568853 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b3a3bf-adfa-42d6-9790-5e7c3337b334 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:07:49.598039 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:07:49.643917 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88694ca-0113-4534-835f-f2d8af868cc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:07:49.644432 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88694ca-0113-4534-835f-f2d8af868cc1 service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:07:49.644708 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88694ca-0113-4534-835f-f2d8af868cc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:07:49.644978 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88694ca-0113-4534-835f-f2d8af868cc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:07:49.646598 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-a88694ca-0113-4534-835f-f2d8af868cc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:07:49.756734 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88694ca-0113-4534-835f-f2d8af868cc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:07:49.758143 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88694ca-0113-4534-835f-f2d8af868cc1 service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:07:49.777418 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 729/1458] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:07:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:07:51.998858 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db52c59-bfa0-46f6-954f-341d5c9f5344 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:07:51.999447 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db52c59-bfa0-46f6-954f-341d5c9f5344 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:07:51.999447 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db52c59-bfa0-46f6-954f-341d5c9f5344 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:07:52.000765 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 730/1459] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 16:07:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:07:52.007491 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11f3952-4eea-4297-8c64-94c5c43acee7 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:07:52.007982 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11f3952-4eea-4297-8c64-94c5c43acee7 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:07:52.008118 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11f3952-4eea-4297-8c64-94c5c43acee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:07:52.075675 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-d11f3952-4eea-4297-8c64-94c5c43acee7 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:07:52.076800 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-d11f3952-4eea-4297-8c64-94c5c43acee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:07:52.085076 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:07:52.180864 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 730/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 16:07:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:07:52.463163 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db52c59-bfa0-46f6-954f-341d5c9f5344 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:07:52.464697 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-5db52c59-bfa0-46f6-954f-341d5c9f5344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:07:52.578381 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db52c59-bfa0-46f6-954f-341d5c9f5344 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:07:52.636791 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:07:52.656245 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaba903-13c2-4ef6-be94-59a5775b05a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:07:52.656849 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaba903-13c2-4ef6-be94-59a5775b05a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:07:52.657121 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaba903-13c2-4ef6-be94-59a5775b05a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:07:52.657387 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaba903-13c2-4ef6-be94-59a5775b05a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:07:52.659025 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-caaba903-13c2-4ef6-be94-59a5775b05a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:07:52.774727 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaba903-13c2-4ef6-be94-59a5775b05a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:07:52.776103 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaba903-13c2-4ef6-be94-59a5775b05a5 service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:07:52.795026 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 731/1461] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:07:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 16:07:55.070073 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eada19-c588-4d8c-9a39-3884ad77d64a None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:07:55.070638 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eada19-c588-4d8c-9a39-3884ad77d64a None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:07:55.070638 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eada19-c588-4d8c-9a39-3884ad77d64a None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:07:55.071881 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 731/1462] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 16:07:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:07:55.077302 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54261f1e-c540-4a49-919f-7e73cd531d3f None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:07:55.077778 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54261f1e-c540-4a49-919f-7e73cd531d3f None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:07:55.077891 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54261f1e-c540-4a49-919f-7e73cd531d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:07:55.147758 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-54261f1e-c540-4a49-919f-7e73cd531d3f None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:07:55.148978 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-54261f1e-c540-4a49-919f-7e73cd531d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:07:55.189486 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:07:55.250778 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 732/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 16:07:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:07:55.544123 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eada19-c588-4d8c-9a39-3884ad77d64a None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:07:55.545657 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-32eada19-c588-4d8c-9a39-3884ad77d64a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:07:55.660343 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eada19-c588-4d8c-9a39-3884ad77d64a None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:07:55.700705 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:07:55.736930 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee54743d-4eaf-4e74-9cab-8abe1e18d6e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:07:55.737533 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee54743d-4eaf-4e74-9cab-8abe1e18d6e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:07:55.737810 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee54743d-4eaf-4e74-9cab-8abe1e18d6e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:07:55.738078 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee54743d-4eaf-4e74-9cab-8abe1e18d6e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:07:55.739710 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-ee54743d-4eaf-4e74-9cab-8abe1e18d6e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:07:55.850503 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee54743d-4eaf-4e74-9cab-8abe1e18d6e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:07:55.851940 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee54743d-4eaf-4e74-9cab-8abe1e18d6e1 service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:07:55.870030 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 732/1464] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:07:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:07:58.097045 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2900956d-bc76-45b7-af7f-f641845b6f6c None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:07:58.097613 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2900956d-bc76-45b7-af7f-f641845b6f6c None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:07:58.097613 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2900956d-bc76-45b7-af7f-f641845b6f6c None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:07:58.098955 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 733/1465] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 16:07:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:07:58.104397 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7faf14f-23a6-4cba-adc5-87e2c5a38b11 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:07:58.104844 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7faf14f-23a6-4cba-adc5-87e2c5a38b11 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:07:58.104937 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7faf14f-23a6-4cba-adc5-87e2c5a38b11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:07:58.171393 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-a7faf14f-23a6-4cba-adc5-87e2c5a38b11 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:07:58.172874 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-a7faf14f-23a6-4cba-adc5-87e2c5a38b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:07:58.190571 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:07:58.273244 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 733/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 16:07:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:07:58.562954 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2900956d-bc76-45b7-af7f-f641845b6f6c None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:07:58.566043 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-2900956d-bc76-45b7-af7f-f641845b6f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:07:58.693842 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2900956d-bc76-45b7-af7f-f641845b6f6c None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:07:58.762824 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:07:58.770495 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf90710f-0fd1-4985-a385-a8b37bca12a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:07:58.771108 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf90710f-0fd1-4985-a385-a8b37bca12a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:07:58.771387 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf90710f-0fd1-4985-a385-a8b37bca12a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:07:58.771657 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf90710f-0fd1-4985-a385-a8b37bca12a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:07:58.773338 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-bf90710f-0fd1-4985-a385-a8b37bca12a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:07:58.887235 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf90710f-0fd1-4985-a385-a8b37bca12a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:07:58.888582 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf90710f-0fd1-4985-a385-a8b37bca12a0 service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:07:58.906929 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 734/1467] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:07:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:08:01.129088 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a3b742-7924-4b91-bcee-61098c3c1cf8 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:08:01.129642 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a3b742-7924-4b91-bcee-61098c3c1cf8 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:08:01.129642 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a3b742-7924-4b91-bcee-61098c3c1cf8 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:08:01.130904 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 734/1468] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 16:08:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:08:01.136259 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a98a9c-38f0-488d-81db-beece988f3cf None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:08:01.136712 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a98a9c-38f0-488d-81db-beece988f3cf None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:08:01.136821 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a98a9c-38f0-488d-81db-beece988f3cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:08:01.203066 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-76a98a9c-38f0-488d-81db-beece988f3cf None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:08:01.204327 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-76a98a9c-38f0-488d-81db-beece988f3cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:08:01.256488 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:08:01.310980 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 735/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 16:08:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:08:01.594864 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a3b742-7924-4b91-bcee-61098c3c1cf8 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:08:01.597095 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-d4a3b742-7924-4b91-bcee-61098c3c1cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:08:01.724586 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a3b742-7924-4b91-bcee-61098c3c1cf8 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:08:01.773965 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:08:01.800343 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe189ef9-9f1b-41c8-b6f8-a4864e821d77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:08:01.800946 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe189ef9-9f1b-41c8-b6f8-a4864e821d77 service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:08:01.801215 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe189ef9-9f1b-41c8-b6f8-a4864e821d77 service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:08:01.801513 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe189ef9-9f1b-41c8-b6f8-a4864e821d77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:08:01.803119 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-fe189ef9-9f1b-41c8-b6f8-a4864e821d77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:08:01.914604 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe189ef9-9f1b-41c8-b6f8-a4864e821d77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:08:01.915964 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe189ef9-9f1b-41c8-b6f8-a4864e821d77 service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:08:01.934059 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 735/1470] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:08:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:08:11.311360 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=22 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:08:11.931864 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=66 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:08:22.067578 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9471052a-4b09-45ae-87cd-dddc671a80b0 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:08:22.068122 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9471052a-4b09-45ae-87cd-dddc671a80b0 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:08:22.068122 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9471052a-4b09-45ae-87cd-dddc671a80b0 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:08:22.069384 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 736/1471] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:08:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:08:22.074927 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70874bcf-3283-4417-a037-e9aabbee6a93 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:08:22.075361 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70874bcf-3283-4417-a037-e9aabbee6a93 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:08:22.075458 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70874bcf-3283-4417-a037-e9aabbee6a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:08:22.144639 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-70874bcf-3283-4417-a037-e9aabbee6a93 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:08:22.145856 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-70874bcf-3283-4417-a037-e9aabbee6a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:08:22.244493 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 736/1472] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:08:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:08:22.547903 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b2aa92-9520-4239-841f-a6d3c81f0990 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:08:22.548218 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b2aa92-9520-4239-841f-a6d3c81f0990 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:08:22.548493 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b2aa92-9520-4239-841f-a6d3c81f0990 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:08:22.549821 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 737/1473] 10.222.0.57 () {54 vars in 962 bytes} [Mon Sep 19 16:08:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:08:22.555017 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f56e936-9273-4001-94a0-56698d1500db None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:08:22.555302 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f56e936-9273-4001-94a0-56698d1500db None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:08:22.555745 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f56e936-9273-4001-94a0-56698d1500db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:08:22.638828 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-3f56e936-9273-4001-94a0-56698d1500db None None] MFA Rules not processed for user `5632ade0cc6f4ae28f72160de68486cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:08:22.640072 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-3f56e936-9273-4001-94a0-56698d1500db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:08:22.663832 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:08:22.725329 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 737/1474] 10.222.0.57 () {58 vars in 1059 bytes} [Mon Sep 19 16:08:22 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:08:22.733473 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b2aa92-9520-4239-841f-a6d3c81f0990 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:08:22.735718 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-b0b2aa92-9520-4239-841f-a6d3c81f0990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:08:22.845267 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b2aa92-9520-4239-841f-a6d3c81f0990 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:08:22.920514 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4835a20-4c75-420a-bad7-2887ad3f0c3a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5632ade0cc6f4ae28f72160de68486cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:08:22.921119 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4835a20-4c75-420a-bad7-2887ad3f0c3a service glance] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:08:22.921397 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4835a20-4c75-420a-bad7-2887ad3f0c3a service glance] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:08:22.921663 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4835a20-4c75-420a-bad7-2887ad3f0c3a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:08:22.923594 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-f4835a20-4c75-420a-bad7-2887ad3f0c3a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:08:22.983618 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:08:23.040708 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4835a20-4c75-420a-bad7-2887ad3f0c3a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:08:23.042065 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4835a20-4c75-420a-bad7-2887ad3f0c3a service glance] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:08:23.061136 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 738/1475] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:08:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:08:25.351898 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe0262-ea68-4b7f-82cc-63b8f8ad4d5b None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:08:25.352466 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe0262-ea68-4b7f-82cc-63b8f8ad4d5b None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:08:25.352466 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe0262-ea68-4b7f-82cc-63b8f8ad4d5b None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:08:25.353764 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 738/1476] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:08:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:08:25.359303 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26dca4f-da64-416d-a54c-242071c82d46 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:08:25.359742 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26dca4f-da64-416d-a54c-242071c82d46 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:08:25.359860 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26dca4f-da64-416d-a54c-242071c82d46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:08:25.429401 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-c26dca4f-da64-416d-a54c-242071c82d46 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:08:25.430670 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-c26dca4f-da64-416d-a54c-242071c82d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:08:25.530177 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 739/1477] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:08:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:08:25.807305 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fe0262-ea68-4b7f-82cc-63b8f8ad4d5b None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:08:25.808826 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-86fe0262-ea68-4b7f-82cc-63b8f8ad4d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:08:25.934835 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fe0262-ea68-4b7f-82cc-63b8f8ad4d5b None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:08:25.996231 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:08:26.047927 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c5c2ea-8863-44ff-afa7-35adc2ab628b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:08:26.048456 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c5c2ea-8863-44ff-afa7-35adc2ab628b admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:08:26.048738 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c5c2ea-8863-44ff-afa7-35adc2ab628b admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:08:26.049011 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c5c2ea-8863-44ff-afa7-35adc2ab628b admin admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:08:26.049657 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c5c2ea-8863-44ff-afa7-35adc2ab628b admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:08:26.051418 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: WARNING py.warnings [None req-d3c5c2ea-8863-44ff-afa7-35adc2ab628b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 16:08:26.051418 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: warnings.warn(msg) Sep 19 16:08:26.051418 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: Sep 19 16:08:26.052560 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c5c2ea-8863-44ff-afa7-35adc2ab628b admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:08:26.072238 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 739/1478] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 19 16:08:25 2022] GET /identity/v3/projects => generated 2553 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:08:28.294296 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cb2e73-c02c-4450-a247-f620847719f1 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:08:28.294897 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cb2e73-c02c-4450-a247-f620847719f1 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:08:28.294897 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cb2e73-c02c-4450-a247-f620847719f1 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:08:28.296200 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 740/1479] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:08:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:08:28.301986 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd8a106-fd0a-4ea9-becc-89312a609c10 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:08:28.302463 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd8a106-fd0a-4ea9-becc-89312a609c10 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:08:28.302580 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd8a106-fd0a-4ea9-becc-89312a609c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:08:28.372550 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-1fd8a106-fd0a-4ea9-becc-89312a609c10 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:08:28.373772 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-1fd8a106-fd0a-4ea9-becc-89312a609c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:08:28.471356 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 740/1480] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:08:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:08:28.769133 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cb2e73-c02c-4450-a247-f620847719f1 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:08:28.770759 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-d9cb2e73-c02c-4450-a247-f620847719f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:08:28.781222 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:08:28.924386 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cb2e73-c02c-4450-a247-f620847719f1 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:08:29.003460 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:08:29.038255 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ae2fa4-ab65-490b-8b9b-4783ea26470f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:08:29.038802 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ae2fa4-ab65-490b-8b9b-4783ea26470f service nova] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:08:29.039079 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ae2fa4-ab65-490b-8b9b-4783ea26470f service nova] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:08:29.039349 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ae2fa4-ab65-490b-8b9b-4783ea26470f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:08:29.040958 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-b5ae2fa4-ab65-490b-8b9b-4783ea26470f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:08:29.136309 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ae2fa4-ab65-490b-8b9b-4783ea26470f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:08:29.137911 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ae2fa4-ab65-490b-8b9b-4783ea26470f service nova] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:08:29.157135 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 741/1481] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:08:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:08:31.550257 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdcfbf6-d2b1-49de-8f5e-27160819411f None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:08:31.550869 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdcfbf6-d2b1-49de-8f5e-27160819411f None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:08:31.550869 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdcfbf6-d2b1-49de-8f5e-27160819411f None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:08:31.552183 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 741/1482] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:08:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:08:31.557522 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce9a3e-6855-40a0-b646-db8a6454e499 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:08:31.557956 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce9a3e-6855-40a0-b646-db8a6454e499 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:08:31.558067 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce9a3e-6855-40a0-b646-db8a6454e499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:08:31.625005 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-12ce9a3e-6855-40a0-b646-db8a6454e499 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:08:31.626249 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-12ce9a3e-6855-40a0-b646-db8a6454e499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:08:31.674300 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:08:31.731310 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 742/1483] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:08:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:08:32.030993 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdcfbf6-d2b1-49de-8f5e-27160819411f None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:08:32.033202 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-9fdcfbf6-d2b1-49de-8f5e-27160819411f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:08:32.057466 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:08:32.139074 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdcfbf6-d2b1-49de-8f5e-27160819411f None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:08:32.212336 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45c215e-5b23-4b89-9c8a-36d09d9d24fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:08:32.212938 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c215e-5b23-4b89-9c8a-36d09d9d24fa service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:08:32.213213 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c215e-5b23-4b89-9c8a-36d09d9d24fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:08:32.213479 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c215e-5b23-4b89-9c8a-36d09d9d24fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:08:32.215093 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-b45c215e-5b23-4b89-9c8a-36d09d9d24fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:08:32.293877 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:08:32.330220 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45c215e-5b23-4b89-9c8a-36d09d9d24fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:08:32.331673 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45c215e-5b23-4b89-9c8a-36d09d9d24fa service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:08:32.350504 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 742/1484] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:08:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:08:34.572538 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b397b528-3232-4476-9ef4-27cd01589836 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:08:34.573117 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b397b528-3232-4476-9ef4-27cd01589836 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:08:34.573117 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b397b528-3232-4476-9ef4-27cd01589836 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:08:34.574422 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 743/1485] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:08:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:08:34.580192 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904fd9bc-abc8-415b-a5c1-8055ce42ce64 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:08:34.580606 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904fd9bc-abc8-415b-a5c1-8055ce42ce64 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:08:34.580745 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904fd9bc-abc8-415b-a5c1-8055ce42ce64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:08:34.645720 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-904fd9bc-abc8-415b-a5c1-8055ce42ce64 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:08:34.646963 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-904fd9bc-abc8-415b-a5c1-8055ce42ce64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:08:34.742391 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 743/1486] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:08:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:08:35.032626 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b397b528-3232-4476-9ef4-27cd01589836 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:08:35.034182 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-b397b528-3232-4476-9ef4-27cd01589836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:08:35.139587 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b397b528-3232-4476-9ef4-27cd01589836 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:08:35.217976 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:08:35.220260 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d2dc2d-d2e8-4b9c-8535-3291eb230bce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:08:35.220995 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d2dc2d-d2e8-4b9c-8535-3291eb230bce service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:08:35.221293 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d2dc2d-d2e8-4b9c-8535-3291eb230bce service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:08:35.221658 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d2dc2d-d2e8-4b9c-8535-3291eb230bce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:08:35.223979 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-d6d2dc2d-d2e8-4b9c-8535-3291eb230bce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:08:35.337344 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d2dc2d-d2e8-4b9c-8535-3291eb230bce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:08:35.338801 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d2dc2d-d2e8-4b9c-8535-3291eb230bce service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:08:35.357018 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 744/1487] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:08:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:08:37.677978 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff548c-5ad1-43cd-99a8-be68b4d183d9 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:08:37.678528 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff548c-5ad1-43cd-99a8-be68b4d183d9 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:08:37.678528 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff548c-5ad1-43cd-99a8-be68b4d183d9 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:08:37.679801 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 744/1488] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:08:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:08:37.685360 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d1d637-215d-4a62-b5fd-7db29b48364e None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:08:37.685816 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d1d637-215d-4a62-b5fd-7db29b48364e None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:08:37.685910 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d1d637-215d-4a62-b5fd-7db29b48364e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:08:37.755884 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-71d1d637-215d-4a62-b5fd-7db29b48364e None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:08:37.757117 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-71d1d637-215d-4a62-b5fd-7db29b48364e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:08:37.816025 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:08:37.865613 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 745/1489] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:08:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:08:38.154970 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ff548c-5ad1-43cd-99a8-be68b4d183d9 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:08:38.156738 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-e2ff548c-5ad1-43cd-99a8-be68b4d183d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:08:38.194180 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:08:38.266780 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ff548c-5ad1-43cd-99a8-be68b4d183d9 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:08:38.342788 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9079604e-5daf-4439-8325-cc94bb734e26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:08:38.343399 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9079604e-5daf-4439-8325-cc94bb734e26 service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:08:38.344223 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9079604e-5daf-4439-8325-cc94bb734e26 service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:08:38.344549 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9079604e-5daf-4439-8325-cc94bb734e26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:08:38.346224 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-9079604e-5daf-4439-8325-cc94bb734e26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:08:38.436562 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:08:38.466082 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9079604e-5daf-4439-8325-cc94bb734e26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:08:38.467536 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9079604e-5daf-4439-8325-cc94bb734e26 service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:08:38.489414 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 745/1490] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:08:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:08:40.990695 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f8ef48-a8f4-4fa1-ba70-287b6676b843 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:08:40.991258 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f8ef48-a8f4-4fa1-ba70-287b6676b843 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:08:40.991258 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f8ef48-a8f4-4fa1-ba70-287b6676b843 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:08:40.992630 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 746/1491] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 19 16:08:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:08:40.998162 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafbf855-a995-4405-a963-f45ee82eea3f None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:08:40.998665 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafbf855-a995-4405-a963-f45ee82eea3f None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:08:40.998775 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafbf855-a995-4405-a963-f45ee82eea3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:08:41.081622 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-dafbf855-a995-4405-a963-f45ee82eea3f None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:08:41.082797 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-dafbf855-a995-4405-a963-f45ee82eea3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:08:41.191419 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 746/1492] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 19 16:08:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:08:41.481376 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f8ef48-a8f4-4fa1-ba70-287b6676b843 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:08:41.483876 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-70f8ef48-a8f4-4fa1-ba70-287b6676b843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:08:41.601804 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f8ef48-a8f4-4fa1-ba70-287b6676b843 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:08:41.684314 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf70450-04e8-4553-b69d-b810fa6516e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:08:41.684837 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf70450-04e8-4553-b69d-b810fa6516e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:08:41.685115 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf70450-04e8-4553-b69d-b810fa6516e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:08:41.685388 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf70450-04e8-4553-b69d-b810fa6516e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:08:41.687061 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-7bf70450-04e8-4553-b69d-b810fa6516e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:08:41.696567 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:08:41.817268 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf70450-04e8-4553-b69d-b810fa6516e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:08:41.818858 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf70450-04e8-4553-b69d-b810fa6516e7 service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:08:41.843440 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 747/1493] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:08:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:08:51.187788 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=82 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:08:51.840861 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=50 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:09:56.788316 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f174baa8-6fe5-4f9a-b05f-8d2ca1c8eb9e None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:09:56.788933 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f174baa8-6fe5-4f9a-b05f-8d2ca1c8eb9e None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:09:56.788933 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f174baa8-6fe5-4f9a-b05f-8d2ca1c8eb9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:09:56.872055 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-f174baa8-6fe5-4f9a-b05f-8d2ca1c8eb9e None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:09:56.873937 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-f174baa8-6fe5-4f9a-b05f-8d2ca1c8eb9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:09:56.975696 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 747/1494] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:09:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:09:56.984891 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0d6ca-b645-4188-acf9-135a616ad2b4 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:09:56.985534 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0d6ca-b645-4188-acf9-135a616ad2b4 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:09:56.985534 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0d6ca-b645-4188-acf9-135a616ad2b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:09:57.065325 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-6db0d6ca-b645-4188-acf9-135a616ad2b4 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:09:57.066516 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-6db0d6ca-b645-4188-acf9-135a616ad2b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:09:57.194786 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 748/1495] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:09:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:09:57.199540 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f174baa8-6fe5-4f9a-b05f-8d2ca1c8eb9e None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:09:57.200767 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-f174baa8-6fe5-4f9a-b05f-8d2ca1c8eb9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:09:57.212754 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f174baa8-6fe5-4f9a-b05f-8d2ca1c8eb9e None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:09:57.279233 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:09:57.313948 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47590f4a-e4e4-44f3-adb8-5a92d66ce15b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:09:57.314612 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47590f4a-e4e4-44f3-adb8-5a92d66ce15b admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:09:57.314858 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47590f4a-e4e4-44f3-adb8-5a92d66ce15b admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:09:57.315125 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47590f4a-e4e4-44f3-adb8-5a92d66ce15b admin admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:09:57.315848 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47590f4a-e4e4-44f3-adb8-5a92d66ce15b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:09:57.316214 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47590f4a-e4e4-44f3-adb8-5a92d66ce15b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:09:57.317296 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: WARNING py.warnings [None req-47590f4a-e4e4-44f3-adb8-5a92d66ce15b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 19 16:09:57.317296 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: warnings.warn(msg) Sep 19 16:09:57.317296 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: Sep 19 16:09:57.318325 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47590f4a-e4e4-44f3-adb8-5a92d66ce15b admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:09:57.332369 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 748/1496] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:09:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:09:57.341324 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db0d6ca-b645-4188-acf9-135a616ad2b4 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:09:57.351279 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db0d6ca-b645-4188-acf9-135a616ad2b4 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:09:57.414231 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:09:57.449415 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3934263d-0d71-4105-a03f-4cae504f744b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:09:57.450013 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3934263d-0d71-4105-a03f-4cae504f744b admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:09:57.450291 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3934263d-0d71-4105-a03f-4cae504f744b admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:09:57.450590 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3934263d-0d71-4105-a03f-4cae504f744b admin admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:09:57.451726 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3934263d-0d71-4105-a03f-4cae504f744b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:09:57.453666 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: WARNING py.warnings [None req-3934263d-0d71-4105-a03f-4cae504f744b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 16:09:57.453666 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: warnings.warn(msg) Sep 19 16:09:57.453666 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: Sep 19 16:09:57.454884 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3934263d-0d71-4105-a03f-4cae504f744b admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:09:57.515922 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 749/1497] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:09:57 2022] POST /identity/v3/projects => generated 377 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:09:57.520144 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47590f4a-e4e4-44f3-adb8-5a92d66ce15b admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:09:57.521674 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-47590f4a-e4e4-44f3-adb8-5a92d66ce15b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:09:57.555336 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47590f4a-e4e4-44f3-adb8-5a92d66ce15b admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:09:57.649121 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ee8c57-85e0-4eb2-9d67-7d77ccae0b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:09:57.649642 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee8c57-85e0-4eb2-9d67-7d77ccae0b79 admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:09:57.649921 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee8c57-85e0-4eb2-9d67-7d77ccae0b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:09:57.650196 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee8c57-85e0-4eb2-9d67-7d77ccae0b79 admin admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:09:57.650868 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ee8c57-85e0-4eb2-9d67-7d77ccae0b79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:09:57.652150 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: WARNING py.warnings [None req-85ee8c57-85e0-4eb2-9d67-7d77ccae0b79 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 16:09:57.652150 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: warnings.warn(msg) Sep 19 16:09:57.652150 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: Sep 19 16:09:57.653084 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ee8c57-85e0-4eb2-9d67-7d77ccae0b79 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:09:57.698809 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 749/1498] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:09:57 2022] POST /identity/v3/users => generated 289 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:09:57.703481 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3934263d-0d71-4105-a03f-4cae504f744b admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:09:57.705041 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-3934263d-0d71-4105-a03f-4cae504f744b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:09:57.772036 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3934263d-0d71-4105-a03f-4cae504f744b admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:09:57.864012 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:09:57.878882 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a178167-1147-46ac-aeb6-3a86e681dcc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:09:57.879498 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a178167-1147-46ac-aeb6-3a86e681dcc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:09:57.879774 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a178167-1147-46ac-aeb6-3a86e681dcc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:09:57.880044 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a178167-1147-46ac-aeb6-3a86e681dcc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:09:57.880694 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a178167-1147-46ac-aeb6-3a86e681dcc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:09:57.881721 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: WARNING py.warnings [None req-3a178167-1147-46ac-aeb6-3a86e681dcc1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 19 16:09:57.881721 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: warnings.warn(msg) Sep 19 16:09:57.881721 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: Sep 19 16:09:57.882650 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a178167-1147-46ac-aeb6-3a86e681dcc1 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:09:57.896466 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 750/1499] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:09:57 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 16:09:57.900878 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ee8c57-85e0-4eb2-9d67-7d77ccae0b79 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:09:57.902448 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-85ee8c57-85e0-4eb2-9d67-7d77ccae0b79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:09:57.916920 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ee8c57-85e0-4eb2-9d67-7d77ccae0b79 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:09:57.942659 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:09:58.012920 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57ae070-dfbb-45d9-8159-982d5d48777b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:09:58.013527 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57ae070-dfbb-45d9-8159-982d5d48777b admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:09:58.014487 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57ae070-dfbb-45d9-8159-982d5d48777b admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:09:58.014767 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57ae070-dfbb-45d9-8159-982d5d48777b admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:09:58.015405 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57ae070-dfbb-45d9-8159-982d5d48777b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:09:58.016414 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: WARNING py.warnings [None req-a57ae070-dfbb-45d9-8159-982d5d48777b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 19 16:09:58.016414 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: warnings.warn(msg) Sep 19 16:09:58.016414 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: Sep 19 16:09:58.017297 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57ae070-dfbb-45d9-8159-982d5d48777b admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:09:58.029666 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 750/1500] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:09:57 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:09:58.033862 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a178167-1147-46ac-aeb6-3a86e681dcc1 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:09:58.035422 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-3a178167-1147-46ac-aeb6-3a86e681dcc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:09:58.069536 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a178167-1147-46ac-aeb6-3a86e681dcc1 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:09:58.165051 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc36f6b3-a6d9-4018-8746-3a8273484cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:09:58.165623 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36f6b3-a6d9-4018-8746-3a8273484cb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:09:58.165903 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36f6b3-a6d9-4018-8746-3a8273484cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:09:58.166179 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36f6b3-a6d9-4018-8746-3a8273484cb2 admin admin] PATH_INFO: `/v3/projects/d845cc0e00f94ee8b8354b2eb1503b30/users/b96f787e913a4b6d80ddd91cd809c63b/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:09:58.189737 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc36f6b3-a6d9-4018-8746-3a8273484cb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d845cc0e00f94ee8b8354b2eb1503b30, user_id=b96f787e913a4b6d80ddd91cd809c63b, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:09:58.190783 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: WARNING py.warnings [None req-fc36f6b3-a6d9-4018-8746-3a8273484cb2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 16:09:58.190783 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: warnings.warn(msg) Sep 19 16:09:58.190783 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: Sep 19 16:09:58.192731 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc36f6b3-a6d9-4018-8746-3a8273484cb2 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:09:58.217292 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 751/1501] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:09:58 2022] PUT /identity/v3/projects/d845cc0e00f94ee8b8354b2eb1503b30/users/b96f787e913a4b6d80ddd91cd809c63b/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:09:58.229410 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cead229-25e4-436d-b4eb-cc528816a621 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:09:58.229697 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cead229-25e4-436d-b4eb-cc528816a621 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:09:58.229971 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cead229-25e4-436d-b4eb-cc528816a621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:09:58.286814 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-9cead229-25e4-436d-b4eb-cc528816a621 None None] MFA Rules not processed for user `b96f787e913a4b6d80ddd91cd809c63b`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:09:58.288043 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-9cead229-25e4-436d-b4eb-cc528816a621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:09:58.368676 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 751/1502] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:09:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:09:58.382455 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed8ae14-c6e8-4015-9d91-31d63e98d357 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:09:58.382744 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed8ae14-c6e8-4015-9d91-31d63e98d357 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:09:58.383058 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed8ae14-c6e8-4015-9d91-31d63e98d357 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:09:58.384374 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 752/1503] 10.222.0.57 () {54 vars in 962 bytes} [Mon Sep 19 16:09:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:09:58.391315 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2fda2e-6ea4-4030-a2ea-6f91882360d0 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:09:58.391729 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2fda2e-6ea4-4030-a2ea-6f91882360d0 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:09:58.392107 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2fda2e-6ea4-4030-a2ea-6f91882360d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:09:58.430041 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:09:58.475378 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-0b2fda2e-6ea4-4030-a2ea-6f91882360d0 None None] MFA Rules not processed for user `7d596e43cc7446d29e2d62d9052d6091`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:09:58.476582 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-0b2fda2e-6ea4-4030-a2ea-6f91882360d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:09:58.553092 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 752/1504] 10.222.0.57 () {58 vars in 1059 bytes} [Mon Sep 19 16:09:58 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:09:58.562757 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed8ae14-c6e8-4015-9d91-31d63e98d357 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:09:58.564339 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-eed8ae14-c6e8-4015-9d91-31d63e98d357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:09:58.616777 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:09:58.674972 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed8ae14-c6e8-4015-9d91-31d63e98d357 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:09:58.751049 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168526b4-537d-4e35-9750-fe74078146df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:09:58.751660 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168526b4-537d-4e35-9750-fe74078146df service cinder] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:09:58.751941 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168526b4-537d-4e35-9750-fe74078146df service cinder] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:09:58.752217 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168526b4-537d-4e35-9750-fe74078146df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:09:58.753844 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-168526b4-537d-4e35-9750-fe74078146df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:09:58.853661 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168526b4-537d-4e35-9750-fe74078146df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:09:58.855124 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168526b4-537d-4e35-9750-fe74078146df service cinder] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:09:58.859369 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:09:58.874421 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 753/1505] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:09:58 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:09:58.920830 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745bad29-864b-42ca-b138-3638c8f3c567 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:09:58.921240 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745bad29-864b-42ca-b138-3638c8f3c567 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:09:58.921382 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745bad29-864b-42ca-b138-3638c8f3c567 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:09:58.922673 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 753/1506] 10.222.0.57 () {52 vars in 830 bytes} [Mon Sep 19 16:09:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:09:58.926484 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168526b4-537d-4e35-9750-fe74078146df service cinder] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:09:58.927763 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-168526b4-537d-4e35-9750-fe74078146df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:09:59.027354 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168526b4-537d-4e35-9750-fe74078146df service cinder] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:09:59.120081 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf0c3c7-aaf0-421d-a500-213caff369c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:09:59.120794 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf0c3c7-aaf0-421d-a500-213caff369c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:09:59.120963 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf0c3c7-aaf0-421d-a500-213caff369c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:09:59.121234 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf0c3c7-aaf0-421d-a500-213caff369c1 admin admin] PATH_INFO: `/v3/users/b96f787e913a4b6d80ddd91cd809c63b` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:09:59.133836 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf0c3c7-aaf0-421d-a500-213caff369c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b96f787e913a4b6d80ddd91cd809c63b)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:09:59.134897 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: WARNING py.warnings [None req-bbf0c3c7-aaf0-421d-a500-213caff369c1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 16:09:59.134897 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: warnings.warn(msg) Sep 19 16:09:59.134897 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: Sep 19 16:09:59.136102 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf0c3c7-aaf0-421d-a500-213caff369c1 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:09:59.146466 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:09:59.203377 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-bbf0c3c7-aaf0-421d-a500-213caff369c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b96f787e913a4b6d80ddd91cd809c63b'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:09:59.203821 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-bbf0c3c7-aaf0-421d-a500-213caff369c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b96f787e913a4b6d80ddd91cd809c63b'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:09:59.217872 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-bbf0c3c7-aaf0-421d-a500-213caff369c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b96f787e913a4b6d80ddd91cd809c63b'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:09:59.240006 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-bbf0c3c7-aaf0-421d-a500-213caff369c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b96f787e913a4b6d80ddd91cd809c63b'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:09:59.240572 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-bbf0c3c7-aaf0-421d-a500-213caff369c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b96f787e913a4b6d80ddd91cd809c63b'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:09:59.255610 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 754/1507] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:09:58 2022] DELETE /identity/v3/users/b96f787e913a4b6d80ddd91cd809c63b => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:09:59.259997 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d499c458-4d2c-42b3-9aac-d7abeb1e25de None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:09:59.260443 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d499c458-4d2c-42b3-9aac-d7abeb1e25de None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:09:59.260577 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d499c458-4d2c-42b3-9aac-d7abeb1e25de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:09:59.319393 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-d499c458-4d2c-42b3-9aac-d7abeb1e25de None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:09:59.320536 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-d499c458-4d2c-42b3-9aac-d7abeb1e25de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:09:59.394770 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:09:59.408313 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 754/1508] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:09:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:09:59.418004 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf0c3c7-aaf0-421d-a500-213caff369c1 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:09:59.419327 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-bbf0c3c7-aaf0-421d-a500-213caff369c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:09:59.518820 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf0c3c7-aaf0-421d-a500-213caff369c1 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:09:59.576425 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:09:59.597431 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d8253d-5fa6-471b-9036-5882723dc843 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:09:59.598043 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d8253d-5fa6-471b-9036-5882723dc843 service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:09:59.598318 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d8253d-5fa6-471b-9036-5882723dc843 service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:09:59.598621 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d8253d-5fa6-471b-9036-5882723dc843 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:09:59.600241 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-89d8253d-5fa6-471b-9036-5882723dc843 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:09:59.709452 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d8253d-5fa6-471b-9036-5882723dc843 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:09:59.710926 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d8253d-5fa6-471b-9036-5882723dc843 service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:09:59.729505 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 755/1509] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:09:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:10:00.080216 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d499c458-4d2c-42b3-9aac-d7abeb1e25de None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:10:00.081487 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-d499c458-4d2c-42b3-9aac-d7abeb1e25de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:10:00.093490 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d499c458-4d2c-42b3-9aac-d7abeb1e25de None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:10:00.197826 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a21de2-2c42-437f-9c11-a65a787d9d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:10:00.198602 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a21de2-2c42-437f-9c11-a65a787d9d76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:10:00.199089 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a21de2-2c42-437f-9c11-a65a787d9d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:10:00.199586 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a21de2-2c42-437f-9c11-a65a787d9d76 admin admin] PATH_INFO: `/v3/projects/d845cc0e00f94ee8b8354b2eb1503b30` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:10:00.213957 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a21de2-2c42-437f-9c11-a65a787d9d76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d845cc0e00f94ee8b8354b2eb1503b30)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:10:00.215130 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: WARNING py.warnings [None req-92a21de2-2c42-437f-9c11-a65a787d9d76 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 16:10:00.215130 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: warnings.warn(msg) Sep 19 16:10:00.215130 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: Sep 19 16:10:00.217791 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a21de2-2c42-437f-9c11-a65a787d9d76 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:10:00.289093 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-92a21de2-2c42-437f-9c11-a65a787d9d76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd845cc0e00f94ee8b8354b2eb1503b30', 'request_id': 'req-92a21de2-2c42-437f-9c11-a65a787d9d76'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:10:00.306406 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-92a21de2-2c42-437f-9c11-a65a787d9d76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd845cc0e00f94ee8b8354b2eb1503b30', 'request_id': 'req-92a21de2-2c42-437f-9c11-a65a787d9d76'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:10:00.318800 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-92a21de2-2c42-437f-9c11-a65a787d9d76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd845cc0e00f94ee8b8354b2eb1503b30', 'request_id': 'req-92a21de2-2c42-437f-9c11-a65a787d9d76'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:10:00.345382 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-92a21de2-2c42-437f-9c11-a65a787d9d76 admin admin] The token cache is being invalidate because project d845cc0e00f94ee8b8354b2eb1503b30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81618) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:10:00.345598 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-92a21de2-2c42-437f-9c11-a65a787d9d76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:10:00.345994 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-92a21de2-2c42-437f-9c11-a65a787d9d76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:10:00.347453 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 755/1510] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:10:00 2022] DELETE /identity/v3/projects/d845cc0e00f94ee8b8354b2eb1503b30 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:10:09.727231 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=62 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:10:10.343248 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:11:26.131115 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98052cbf-82cf-4ee7-afd2-0b6be51e1c1d None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:11:26.131696 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98052cbf-82cf-4ee7-afd2-0b6be51e1c1d None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:11:26.131696 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98052cbf-82cf-4ee7-afd2-0b6be51e1c1d None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:11:26.132890 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 756/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 16:11:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:11:26.137677 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3148847-6180-47b4-9dd2-bd1187d9fa3a None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:11:26.138095 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3148847-6180-47b4-9dd2-bd1187d9fa3a None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:11:26.138227 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3148847-6180-47b4-9dd2-bd1187d9fa3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:11:26.221156 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-a3148847-6180-47b4-9dd2-bd1187d9fa3a None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:11:26.222388 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-a3148847-6180-47b4-9dd2-bd1187d9fa3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:11:26.325358 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 756/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 16:11:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:11:26.478126 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98052cbf-82cf-4ee7-afd2-0b6be51e1c1d None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:11:26.480225 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-98052cbf-82cf-4ee7-afd2-0b6be51e1c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:11:26.605836 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98052cbf-82cf-4ee7-afd2-0b6be51e1c1d None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:11:26.613720 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33e0888-42e8-409b-9094-fadb6ab18391 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:11:26.614346 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e0888-42e8-409b-9094-fadb6ab18391 service nova] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:11:26.614661 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e0888-42e8-409b-9094-fadb6ab18391 service nova] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:11:26.614928 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e0888-42e8-409b-9094-fadb6ab18391 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:11:26.623277 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33e0888-42e8-409b-9094-fadb6ab18391 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:11:26.624842 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33e0888-42e8-409b-9094-fadb6ab18391 service nova] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:11:26.634696 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 757/1513] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:11:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:11:27.719355 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df8085-929d-417b-a3a6-4b4315e61213 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:11:27.719925 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df8085-929d-417b-a3a6-4b4315e61213 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:11:27.719925 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df8085-929d-417b-a3a6-4b4315e61213 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:11:27.721300 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 757/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 16:11:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:11:27.727587 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5597b8-efc2-4b8c-a82f-27915d30edcc None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:11:27.728050 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5597b8-efc2-4b8c-a82f-27915d30edcc None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:11:27.728166 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5597b8-efc2-4b8c-a82f-27915d30edcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:11:27.748049 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-7c5597b8-efc2-4b8c-a82f-27915d30edcc None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:11:27.749181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-7c5597b8-efc2-4b8c-a82f-27915d30edcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:11:27.757059 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 758/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 16:11:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:11:27.910309 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94df8085-929d-417b-a3a6-4b4315e61213 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:11:27.926026 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94df8085-929d-417b-a3a6-4b4315e61213 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:11:27.931751 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ec4a7a-c905-4325-ad2e-d16c33ce208c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:11:27.932367 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec4a7a-c905-4325-ad2e-d16c33ce208c service nova] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:11:27.932646 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec4a7a-c905-4325-ad2e-d16c33ce208c service nova] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:11:27.932933 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec4a7a-c905-4325-ad2e-d16c33ce208c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:11:27.941734 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ec4a7a-c905-4325-ad2e-d16c33ce208c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:11:27.943356 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ec4a7a-c905-4325-ad2e-d16c33ce208c service nova] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:11:27.953487 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 758/1516] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:11:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:11:28.556231 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b382c09-32f8-435b-a367-1e33fe7fab62 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:11:28.556609 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b382c09-32f8-435b-a367-1e33fe7fab62 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:11:28.556786 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b382c09-32f8-435b-a367-1e33fe7fab62 None None] PATH_INFO: `/v3` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:11:28.558057 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 759/1517] 10.222.0.16 () {54 vars in 925 bytes} [Mon Sep 19 16:11:28 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:11:28.564801 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e9c818-0d6e-4c16-87a6-af0df35b6ab0 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:11:28.565092 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e9c818-0d6e-4c16-87a6-af0df35b6ab0 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:11:28.565369 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e9c818-0d6e-4c16-87a6-af0df35b6ab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:11:28.626701 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-01e9c818-0d6e-4c16-87a6-af0df35b6ab0 None None] MFA Rules not processed for user `1f1a985495ed48a4acc48e2e6fc7f977`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:11:28.627924 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-01e9c818-0d6e-4c16-87a6-af0df35b6ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:11:28.645548 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:11:28.704986 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 759/1518] 10.222.0.16 () {58 vars in 1015 bytes} [Mon Sep 19 16:11:28 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:11:32.349272 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ee1ddf-48f4-40ae-a013-ea43c9d36646 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:11:32.349742 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ee1ddf-48f4-40ae-a013-ea43c9d36646 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:11:32.349853 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ee1ddf-48f4-40ae-a013-ea43c9d36646 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:11:32.351247 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 760/1519] 10.222.0.57 () {54 vars in 961 bytes} [Mon Sep 19 16:11:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:11:32.356399 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a039f94-d119-4904-aa74-91410f3e8eb2 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:11:32.356853 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a039f94-d119-4904-aa74-91410f3e8eb2 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:11:32.356977 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a039f94-d119-4904-aa74-91410f3e8eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:11:32.379022 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-6a039f94-d119-4904-aa74-91410f3e8eb2 None None] MFA Rules not processed for user `1f1a985495ed48a4acc48e2e6fc7f977`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:11:32.380200 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-6a039f94-d119-4904-aa74-91410f3e8eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:11:32.389312 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 760/1520] 10.222.0.57 () {58 vars in 1058 bytes} [Mon Sep 19 16:11:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:11:32.396988 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ee1ddf-48f4-40ae-a013-ea43c9d36646 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:11:32.410872 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ee1ddf-48f4-40ae-a013-ea43c9d36646 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:11:32.415896 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b144017b-d9c0-46b9-b8dd-d9196508c767 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1a985495ed48a4acc48e2e6fc7f977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:11:32.416484 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b144017b-d9c0-46b9-b8dd-d9196508c767 service placement] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:11:32.416763 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b144017b-d9c0-46b9-b8dd-d9196508c767 service placement] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:11:32.417036 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b144017b-d9c0-46b9-b8dd-d9196508c767 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:11:32.426441 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b144017b-d9c0-46b9-b8dd-d9196508c767 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:11:32.428024 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b144017b-d9c0-46b9-b8dd-d9196508c767 service placement] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:11:32.437829 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 761/1521] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:11:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:11:39.459790 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e06d0a-61e2-4eb6-b0df-09204134dc35 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:11:39.460419 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e06d0a-61e2-4eb6-b0df-09204134dc35 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:11:39.460419 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e06d0a-61e2-4eb6-b0df-09204134dc35 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:11:39.461662 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 761/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 16:11:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:11:39.467924 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5061cf2-1a9e-489d-8175-bf802ca916b6 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:11:39.468448 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5061cf2-1a9e-489d-8175-bf802ca916b6 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:11:39.468448 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5061cf2-1a9e-489d-8175-bf802ca916b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:11:39.488579 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-f5061cf2-1a9e-489d-8175-bf802ca916b6 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:11:39.489786 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-f5061cf2-1a9e-489d-8175-bf802ca916b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:11:39.499628 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 762/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 16:11:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:11:39.653769 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e06d0a-61e2-4eb6-b0df-09204134dc35 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:11:39.669077 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e06d0a-61e2-4eb6-b0df-09204134dc35 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:11:39.674789 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07067025-44d4-4ab1-91d8-e37bed6fb019 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:11:39.675393 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07067025-44d4-4ab1-91d8-e37bed6fb019 service nova] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:11:39.675672 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07067025-44d4-4ab1-91d8-e37bed6fb019 service nova] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:11:39.675948 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07067025-44d4-4ab1-91d8-e37bed6fb019 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:11:39.684204 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07067025-44d4-4ab1-91d8-e37bed6fb019 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:11:39.686287 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07067025-44d4-4ab1-91d8-e37bed6fb019 service nova] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:11:39.695699 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 762/1524] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:11:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:11:40.781371 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47eb475-d768-45c6-b739-95fa8a05d13e None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:11:40.781774 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47eb475-d768-45c6-b739-95fa8a05d13e None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:11:40.781924 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47eb475-d768-45c6-b739-95fa8a05d13e None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:11:40.783302 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 763/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 16:11:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:11:40.789574 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe1464-7904-47f4-aa23-cd2abfc57518 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:11:40.790178 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe1464-7904-47f4-aa23-cd2abfc57518 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:11:40.790178 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe1464-7904-47f4-aa23-cd2abfc57518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:11:40.810013 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-17fe1464-7904-47f4-aa23-cd2abfc57518 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:11:40.811204 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-17fe1464-7904-47f4-aa23-cd2abfc57518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:11:40.819051 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 763/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 16:11:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:11:40.972597 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47eb475-d768-45c6-b739-95fa8a05d13e None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:11:40.987112 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47eb475-d768-45c6-b739-95fa8a05d13e None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:11:40.993521 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f2f1fe-b55e-441e-9ed2-8cad4d2ff66e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:11:40.994119 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f2f1fe-b55e-441e-9ed2-8cad4d2ff66e service nova] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:11:40.994428 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f2f1fe-b55e-441e-9ed2-8cad4d2ff66e service nova] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:11:40.994704 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f2f1fe-b55e-441e-9ed2-8cad4d2ff66e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:11:41.003166 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f2f1fe-b55e-441e-9ed2-8cad4d2ff66e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:11:41.004769 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f2f1fe-b55e-441e-9ed2-8cad4d2ff66e service nova] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:11:41.015511 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 764/1527] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:11:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:11:45.058302 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64748417-0b47-4010-9da6-3bdc3017a4a7 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:11:45.058905 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64748417-0b47-4010-9da6-3bdc3017a4a7 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:11:45.058905 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64748417-0b47-4010-9da6-3bdc3017a4a7 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:11:45.060872 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 764/1528] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 16:11:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:11:45.066654 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04355b5f-2f7d-4411-ad8c-11c1e8829702 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:11:45.067106 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04355b5f-2f7d-4411-ad8c-11c1e8829702 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:11:45.067286 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04355b5f-2f7d-4411-ad8c-11c1e8829702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:11:45.087282 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-04355b5f-2f7d-4411-ad8c-11c1e8829702 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:11:45.088487 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-04355b5f-2f7d-4411-ad8c-11c1e8829702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:11:45.096384 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 765/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 16:11:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:11:45.384508 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64748417-0b47-4010-9da6-3bdc3017a4a7 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:11:45.386227 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-64748417-0b47-4010-9da6-3bdc3017a4a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:11:45.466495 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64748417-0b47-4010-9da6-3bdc3017a4a7 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:11:45.470087 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b745b4ef-0f5c-4c53-97ea-0c5db6dca350 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:11:45.470712 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b745b4ef-0f5c-4c53-97ea-0c5db6dca350 service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:11:45.471101 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b745b4ef-0f5c-4c53-97ea-0c5db6dca350 service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:11:45.471376 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b745b4ef-0f5c-4c53-97ea-0c5db6dca350 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:11:45.480102 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b745b4ef-0f5c-4c53-97ea-0c5db6dca350 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:11:45.481557 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b745b4ef-0f5c-4c53-97ea-0c5db6dca350 service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:11:45.490660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 765/1530] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:11:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:11:47.649836 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c1700-a65b-4227-ae20-a05dd37965f2 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:11:47.650435 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c1700-a65b-4227-ae20-a05dd37965f2 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:11:47.650435 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c1700-a65b-4227-ae20-a05dd37965f2 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:11:47.651779 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 766/1531] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 16:11:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:11:47.657575 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0a8d16-9dc2-4cf2-aee9-879c9fd8f7b1 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:11:47.657988 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0a8d16-9dc2-4cf2-aee9-879c9fd8f7b1 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:11:47.658106 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0a8d16-9dc2-4cf2-aee9-879c9fd8f7b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:11:47.677337 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-9b0a8d16-9dc2-4cf2-aee9-879c9fd8f7b1 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:11:47.678562 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-9b0a8d16-9dc2-4cf2-aee9-879c9fd8f7b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:11:47.686503 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 766/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 16:11:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:11:48.007831 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65c1700-a65b-4227-ae20-a05dd37965f2 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:11:48.009886 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-c65c1700-a65b-4227-ae20-a05dd37965f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:11:48.023813 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65c1700-a65b-4227-ae20-a05dd37965f2 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:11:48.027878 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22fb8d4-de9e-4419-b382-657a5572e734 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:11:48.028833 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22fb8d4-de9e-4419-b382-657a5572e734 service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:11:48.029215 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22fb8d4-de9e-4419-b382-657a5572e734 service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:11:48.029598 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22fb8d4-de9e-4419-b382-657a5572e734 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:11:48.039153 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22fb8d4-de9e-4419-b382-657a5572e734 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:11:48.040584 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22fb8d4-de9e-4419-b382-657a5572e734 service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:11:48.050852 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 767/1533] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:11:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:11:51.727268 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aefd94-7023-40f3-b408-75366e723ceb None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:11:51.727829 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aefd94-7023-40f3-b408-75366e723ceb None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:11:51.727829 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aefd94-7023-40f3-b408-75366e723ceb None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:11:51.729210 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 767/1534] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 16:11:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:11:51.735090 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7163c28c-3884-4c0c-9e27-b143487ce2a6 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:11:51.735742 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7163c28c-3884-4c0c-9e27-b143487ce2a6 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:11:51.735742 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7163c28c-3884-4c0c-9e27-b143487ce2a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:11:51.743465 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=81 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:11:51.757419 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-7163c28c-3884-4c0c-9e27-b143487ce2a6 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:11:51.758612 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-7163c28c-3884-4c0c-9e27-b143487ce2a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:11:51.767265 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 768/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 16:11:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:11:52.089372 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aefd94-7023-40f3-b408-75366e723ceb None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:11:52.091841 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-38aefd94-7023-40f3-b408-75366e723ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:11:52.156818 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:11:52.170542 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aefd94-7023-40f3-b408-75366e723ceb None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:11:52.174590 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a20406-40e8-4359-8546-00afe7c95624 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:11:52.175259 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a20406-40e8-4359-8546-00afe7c95624 service cinder] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:11:52.175572 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a20406-40e8-4359-8546-00afe7c95624 service cinder] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:11:52.175896 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a20406-40e8-4359-8546-00afe7c95624 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:11:52.185665 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a20406-40e8-4359-8546-00afe7c95624 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:11:52.187131 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a20406-40e8-4359-8546-00afe7c95624 service cinder] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:11:52.196828 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 768/1536] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:11:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:11:54.358038 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caad843-042c-4c00-8c38-eb7c5f0c6341 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:11:54.358598 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caad843-042c-4c00-8c38-eb7c5f0c6341 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:11:54.358598 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caad843-042c-4c00-8c38-eb7c5f0c6341 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:11:54.359949 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 769/1537] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 16:11:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:11:54.366059 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01482cd2-e25e-45b0-94ca-0e85e036d17a None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:11:54.366650 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01482cd2-e25e-45b0-94ca-0e85e036d17a None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:11:54.366650 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01482cd2-e25e-45b0-94ca-0e85e036d17a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:11:54.386710 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-01482cd2-e25e-45b0-94ca-0e85e036d17a None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:11:54.388471 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-01482cd2-e25e-45b0-94ca-0e85e036d17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:11:54.396458 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 769/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 16:11:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:11:54.698075 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caad843-042c-4c00-8c38-eb7c5f0c6341 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:11:54.699982 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-6caad843-042c-4c00-8c38-eb7c5f0c6341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:11:54.713322 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caad843-042c-4c00-8c38-eb7c5f0c6341 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:11:54.717419 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7535618d-fbd2-4690-950a-57bcb6d4db2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:11:54.718011 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7535618d-fbd2-4690-950a-57bcb6d4db2a service cinder] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:11:54.718289 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7535618d-fbd2-4690-950a-57bcb6d4db2a service cinder] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:11:54.718593 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7535618d-fbd2-4690-950a-57bcb6d4db2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:11:54.727299 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7535618d-fbd2-4690-950a-57bcb6d4db2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:11:54.728711 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7535618d-fbd2-4690-950a-57bcb6d4db2a service cinder] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:11:54.738644 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 770/1539] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:11:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:12:04.381790 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=11 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:12:04.734561 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=8 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:13:19.116252 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e55596-7f2b-4055-8863-6be4b1d155d4 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:19.118288 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e55596-7f2b-4055-8863-6be4b1d155d4 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:19.118288 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e55596-7f2b-4055-8863-6be4b1d155d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:19.172568 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-38e55596-7f2b-4055-8863-6be4b1d155d4 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:13:19.174629 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-38e55596-7f2b-4055-8863-6be4b1d155d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:19.189219 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 770/1540] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:13:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:13:19.198995 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5303a1-2856-4729-a78c-68279e0a551d None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:19.200017 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5303a1-2856-4729-a78c-68279e0a551d None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:19.200515 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5303a1-2856-4729-a78c-68279e0a551d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:19.232785 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-6d5303a1-2856-4729-a78c-68279e0a551d None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:13:19.234516 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-6d5303a1-2856-4729-a78c-68279e0a551d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:19.245743 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 771/1541] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:13:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:13:19.251710 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e55596-7f2b-4055-8863-6be4b1d155d4 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:19.253651 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd3279f-205d-417a-a2fe-67a7dc23cd64 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:19.254197 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd3279f-205d-417a-a2fe-67a7dc23cd64 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:19.254717 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd3279f-205d-417a-a2fe-67a7dc23cd64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:19.283343 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e55596-7f2b-4055-8863-6be4b1d155d4 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:19.289791 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-edd3279f-205d-417a-a2fe-67a7dc23cd64 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:13:19.292197 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-edd3279f-205d-417a-a2fe-67a7dc23cd64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:19.302626 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e94d2e1-f550-449c-bb2c-919704ff8501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:19.303865 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e94d2e1-f550-449c-bb2c-919704ff8501 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:19.304345 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e94d2e1-f550-449c-bb2c-919704ff8501 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:19.304793 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e94d2e1-f550-449c-bb2c-919704ff8501 admin admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:19.306008 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e94d2e1-f550-449c-bb2c-919704ff8501 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:13:19.307205 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e94d2e1-f550-449c-bb2c-919704ff8501 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:19.310937 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 772/1542] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:13:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:13:19.314670 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e94d2e1-f550-449c-bb2c-919704ff8501 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:19.321427 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38336e6e-54a0-4421-abc1-3c42a6eacf73 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:19.322187 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38336e6e-54a0-4421-abc1-3c42a6eacf73 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:19.322916 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38336e6e-54a0-4421-abc1-3c42a6eacf73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:19.356042 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 771/1543] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:13:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:13:19.365340 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e94d2e1-f550-449c-bb2c-919704ff8501 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:19.371826 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-38336e6e-54a0-4421-abc1-3c42a6eacf73 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:13:19.373575 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-38336e6e-54a0-4421-abc1-3c42a6eacf73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:19.393604 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e94d2e1-f550-449c-bb2c-919704ff8501 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:19.403276 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 773/1544] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:13:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:13:19.407885 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38336e6e-54a0-4421-abc1-3c42a6eacf73 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:19.411734 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec57e907-06de-48c6-a2e2-53db1452bde6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:19.412747 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec57e907-06de-48c6-a2e2-53db1452bde6 admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:19.413222 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec57e907-06de-48c6-a2e2-53db1452bde6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:19.413666 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec57e907-06de-48c6-a2e2-53db1452bde6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:19.414865 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec57e907-06de-48c6-a2e2-53db1452bde6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:19.417338 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: WARNING py.warnings [None req-ec57e907-06de-48c6-a2e2-53db1452bde6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 16:13:19.417338 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: warnings.warn(msg) Sep 19 16:13:19.417338 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: Sep 19 16:13:19.422406 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec57e907-06de-48c6-a2e2-53db1452bde6 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:19.426019 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38336e6e-54a0-4421-abc1-3c42a6eacf73 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:19.435683 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92049e29-7f2b-4104-ae08-04181470be5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:19.436715 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92049e29-7f2b-4104-ae08-04181470be5e admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:19.437211 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92049e29-7f2b-4104-ae08-04181470be5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:19.437702 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92049e29-7f2b-4104-ae08-04181470be5e admin admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:19.438612 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92049e29-7f2b-4104-ae08-04181470be5e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:13:19.439338 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92049e29-7f2b-4104-ae08-04181470be5e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:19.440579 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: WARNING py.warnings [None req-92049e29-7f2b-4104-ae08-04181470be5e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 19 16:13:19.440579 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: warnings.warn(msg) Sep 19 16:13:19.440579 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: Sep 19 16:13:19.441846 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92049e29-7f2b-4104-ae08-04181470be5e admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:19.460845 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 774/1545] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:13:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:13:19.470816 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92049e29-7f2b-4104-ae08-04181470be5e admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:19.482708 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92049e29-7f2b-4104-ae08-04181470be5e admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:19.497543 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e1e9a2-fc9f-4d88-9f3c-3f947c01fec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:19.498719 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e1e9a2-fc9f-4d88-9f3c-3f947c01fec7 admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:19.499578 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e1e9a2-fc9f-4d88-9f3c-3f947c01fec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:19.500130 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e1e9a2-fc9f-4d88-9f3c-3f947c01fec7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:19.501109 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e1e9a2-fc9f-4d88-9f3c-3f947c01fec7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:19.503463 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e1e9a2-fc9f-4d88-9f3c-3f947c01fec7 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:19.529550 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 772/1546] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:13:19 2022] POST /identity/v3/projects => generated 365 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:13:19.534400 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec57e907-06de-48c6-a2e2-53db1452bde6 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:19.561646 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 775/1547] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:13:19 2022] POST /identity/v3/projects => generated 385 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 16:13:19.572227 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e1e9a2-fc9f-4d88-9f3c-3f947c01fec7 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:19.709832 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e1e9a2-fc9f-4d88-9f3c-3f947c01fec7 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:19.724967 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec57e907-06de-48c6-a2e2-53db1452bde6 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:19.736064 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca640e4-ab55-45cc-b182-8d1e4687f7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:19.736438 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3855ec12-cee8-4ba9-8bc2-9e0110369562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:19.736704 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca640e4-ab55-45cc-b182-8d1e4687f7a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:19.736985 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca640e4-ab55-45cc-b182-8d1e4687f7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:19.737257 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca640e4-ab55-45cc-b182-8d1e4687f7a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:19.737426 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3855ec12-cee8-4ba9-8bc2-9e0110369562 admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:19.737896 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca640e4-ab55-45cc-b182-8d1e4687f7a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:19.738021 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3855ec12-cee8-4ba9-8bc2-9e0110369562 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:19.738396 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3855ec12-cee8-4ba9-8bc2-9e0110369562 admin admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:19.739014 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: WARNING py.warnings [None req-7ca640e4-ab55-45cc-b182-8d1e4687f7a9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 16:13:19.739014 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: warnings.warn(msg) Sep 19 16:13:19.739014 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: Sep 19 16:13:19.739666 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3855ec12-cee8-4ba9-8bc2-9e0110369562 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:19.740428 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca640e4-ab55-45cc-b182-8d1e4687f7a9 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:19.745312 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3855ec12-cee8-4ba9-8bc2-9e0110369562 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:19.791365 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 776/1548] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:13:19 2022] POST /identity/v3/users => generated 293 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:13:19.793097 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6613d7d8-eb0d-428e-bb9b-b1fb5b07a8cd None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:19.794983 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6613d7d8-eb0d-428e-bb9b-b1fb5b07a8cd None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:19.795997 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6613d7d8-eb0d-428e-bb9b-b1fb5b07a8cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:19.807055 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 773/1549] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:13:19 2022] POST /identity/v3/users => generated 283 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:13:19.808746 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12319d37-e1e5-4e1b-a588-b90f7f12d0ff None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:19.809272 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12319d37-e1e5-4e1b-a588-b90f7f12d0ff None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:19.809823 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12319d37-e1e5-4e1b-a588-b90f7f12d0ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:19.818925 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-6613d7d8-eb0d-428e-bb9b-b1fb5b07a8cd None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:13:19.820180 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-6613d7d8-eb0d-428e-bb9b-b1fb5b07a8cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:19.830076 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 777/1550] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:13:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:13:19.831677 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572c07af-6145-49a7-bf06-418b7266cb53 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:19.831977 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572c07af-6145-49a7-bf06-418b7266cb53 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:19.832236 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572c07af-6145-49a7-bf06-418b7266cb53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:19.838806 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-12319d37-e1e5-4e1b-a588-b90f7f12d0ff None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:13:19.840391 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-12319d37-e1e5-4e1b-a588-b90f7f12d0ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:19.850612 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 774/1551] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:13:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:13:19.858399 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12319d37-e1e5-4e1b-a588-b90f7f12d0ff None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:19.865403 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-572c07af-6145-49a7-bf06-418b7266cb53 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:13:19.867067 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-572c07af-6145-49a7-bf06-418b7266cb53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:19.873193 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12319d37-e1e5-4e1b-a588-b90f7f12d0ff None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:19.877332 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 778/1552] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:13:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:13:19.878956 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f7507-4c66-4e03-b659-ad3c4c4fc923 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:19.879240 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f7507-4c66-4e03-b659-ad3c4c4fc923 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:19.879514 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f7507-4c66-4e03-b659-ad3c4c4fc923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:19.886763 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b7f6b9-4bdb-471b-b1ca-4b6409cc958d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:19.887818 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b7f6b9-4bdb-471b-b1ca-4b6409cc958d admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:19.888318 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b7f6b9-4bdb-471b-b1ca-4b6409cc958d admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:19.888806 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b7f6b9-4bdb-471b-b1ca-4b6409cc958d admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:19.889984 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b7f6b9-4bdb-471b-b1ca-4b6409cc958d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:19.893087 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b7f6b9-4bdb-471b-b1ca-4b6409cc958d admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:19.903452 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-454f7507-4c66-4e03-b659-ad3c4c4fc923 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:13:19.904694 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-454f7507-4c66-4e03-b659-ad3c4c4fc923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:19.909004 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 775/1553] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:13:19 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:13:19.910092 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b7f6b9-4bdb-471b-b1ca-4b6409cc958d admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:19.926204 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 779/1554] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:13:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:13:19.926204 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71711c3d-7d80-445f-81aa-ddbe4a15bda7 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:19.926204 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71711c3d-7d80-445f-81aa-ddbe4a15bda7 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:19.926204 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71711c3d-7d80-445f-81aa-ddbe4a15bda7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:19.930598 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b7f6b9-4bdb-471b-b1ca-4b6409cc958d admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:19.938457 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5487f70b-9f43-4e20-aff6-34746613f8fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:19.939506 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5487f70b-9f43-4e20-aff6-34746613f8fd admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:19.939977 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5487f70b-9f43-4e20-aff6-34746613f8fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:19.940396 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5487f70b-9f43-4e20-aff6-34746613f8fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:19.941292 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5487f70b-9f43-4e20-aff6-34746613f8fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:19.943311 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5487f70b-9f43-4e20-aff6-34746613f8fd admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:19.951836 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-71711c3d-7d80-445f-81aa-ddbe4a15bda7 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:13:19.953079 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-71711c3d-7d80-445f-81aa-ddbe4a15bda7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:19.960959 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 776/1555] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:13:19 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:13:19.960959 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b01865e-5dfa-408d-8836-3c7e8909f479 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:19.960959 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b01865e-5dfa-408d-8836-3c7e8909f479 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:19.960959 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b01865e-5dfa-408d-8836-3c7e8909f479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:19.967939 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 780/1556] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:13:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:13:19.969609 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab1b629-519d-4ee7-a58b-7eaa3dbdf4ae None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:19.969792 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab1b629-519d-4ee7-a58b-7eaa3dbdf4ae None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:19.970075 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab1b629-519d-4ee7-a58b-7eaa3dbdf4ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:19.990270 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-3b01865e-5dfa-408d-8836-3c7e8909f479 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:13:19.991646 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-3b01865e-5dfa-408d-8836-3c7e8909f479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:19.999016 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-aab1b629-519d-4ee7-a58b-7eaa3dbdf4ae None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:13:20.000266 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-aab1b629-519d-4ee7-a58b-7eaa3dbdf4ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:20.006268 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 777/1557] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:13:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:13:20.007605 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd77d0b5-5a89-44d1-86ff-8977d55d140a None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:20.007900 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd77d0b5-5a89-44d1-86ff-8977d55d140a None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:20.008175 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd77d0b5-5a89-44d1-86ff-8977d55d140a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:20.019409 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 781/1558] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:13:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:13:20.021148 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab1b629-519d-4ee7-a58b-7eaa3dbdf4ae None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:20.029920 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-dd77d0b5-5a89-44d1-86ff-8977d55d140a None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:13:20.031193 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-dd77d0b5-5a89-44d1-86ff-8977d55d140a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:20.036214 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab1b629-519d-4ee7-a58b-7eaa3dbdf4ae None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:20.039469 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 778/1559] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:13:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:13:20.041256 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8887d5c-c4af-430a-9ae1-18be07c08543 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:20.041554 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8887d5c-c4af-430a-9ae1-18be07c08543 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:20.041874 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8887d5c-c4af-430a-9ae1-18be07c08543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:20.042933 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaf0b6e-b239-4af6-a48b-ea1c272fedc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:20.043932 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaf0b6e-b239-4af6-a48b-ea1c272fedc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:20.044414 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaf0b6e-b239-4af6-a48b-ea1c272fedc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:20.044884 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaf0b6e-b239-4af6-a48b-ea1c272fedc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:20.046277 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaf0b6e-b239-4af6-a48b-ea1c272fedc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:20.048517 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaf0b6e-b239-4af6-a48b-ea1c272fedc0 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:20.060041 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 782/1560] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:13:20 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:13:20.060982 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaf0b6e-b239-4af6-a48b-ea1c272fedc0 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:20.063512 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-c8887d5c-c4af-430a-9ae1-18be07c08543 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:13:20.064839 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-c8887d5c-c4af-430a-9ae1-18be07c08543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:20.073390 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 779/1561] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:13:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:13:20.073669 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaf0b6e-b239-4af6-a48b-ea1c272fedc0 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:20.074228 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8887d5c-c4af-430a-9ae1-18be07c08543 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:20.082730 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4251815c-dd68-4da1-8c43-a172078dbf64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:20.083473 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251815c-dd68-4da1-8c43-a172078dbf64 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:20.083852 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251815c-dd68-4da1-8c43-a172078dbf64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:20.084246 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251815c-dd68-4da1-8c43-a172078dbf64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:20.084432 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8887d5c-c4af-430a-9ae1-18be07c08543 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:20.085150 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4251815c-dd68-4da1-8c43-a172078dbf64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:20.087075 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4251815c-dd68-4da1-8c43-a172078dbf64 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:20.089830 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73679c75-aa69-4848-96df-118a929b1bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:20.090556 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73679c75-aa69-4848-96df-118a929b1bf1 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:20.090840 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73679c75-aa69-4848-96df-118a929b1bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:20.091113 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73679c75-aa69-4848-96df-118a929b1bf1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:20.091773 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73679c75-aa69-4848-96df-118a929b1bf1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:13:20.092076 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73679c75-aa69-4848-96df-118a929b1bf1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:20.093674 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73679c75-aa69-4848-96df-118a929b1bf1 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:20.101252 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 783/1562] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:13:20 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:13:20.102968 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4251815c-dd68-4da1-8c43-a172078dbf64 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:20.118060 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 780/1563] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:13:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:13:20.118060 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45646fd5-82b4-40fc-a494-4f1f5db408dc None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:20.118060 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45646fd5-82b4-40fc-a494-4f1f5db408dc None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:20.118060 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45646fd5-82b4-40fc-a494-4f1f5db408dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:20.123817 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4251815c-dd68-4da1-8c43-a172078dbf64 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:20.128979 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377f4f37-da5d-4845-86d7-5d4a5a0f0d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:20.129596 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f4f37-da5d-4845-86d7-5d4a5a0f0d0e admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:20.129878 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f4f37-da5d-4845-86d7-5d4a5a0f0d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:20.130162 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f4f37-da5d-4845-86d7-5d4a5a0f0d0e admin admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:20.130876 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377f4f37-da5d-4845-86d7-5d4a5a0f0d0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:13:20.131202 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377f4f37-da5d-4845-86d7-5d4a5a0f0d0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:20.132813 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377f4f37-da5d-4845-86d7-5d4a5a0f0d0e admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:20.134586 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-45646fd5-82b4-40fc-a494-4f1f5db408dc None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:13:20.135845 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-45646fd5-82b4-40fc-a494-4f1f5db408dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:20.143810 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 781/1564] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:13:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:13:20.145158 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a14b17-281e-4404-9198-5542533f66db None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:20.145908 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a14b17-281e-4404-9198-5542533f66db None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:20.145908 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a14b17-281e-4404-9198-5542533f66db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:20.148487 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 784/1565] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:13:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:13:20.149244 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377f4f37-da5d-4845-86d7-5d4a5a0f0d0e admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:20.171459 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377f4f37-da5d-4845-86d7-5d4a5a0f0d0e admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:20.174711 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-a2a14b17-281e-4404-9198-5542533f66db None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:13:20.176663 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-a2a14b17-281e-4404-9198-5542533f66db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:20.178362 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ad6e05-7756-403c-81f8-2e1a1cad9d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:20.179355 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ad6e05-7756-403c-81f8-2e1a1cad9d15 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:20.179812 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ad6e05-7756-403c-81f8-2e1a1cad9d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:20.180276 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ad6e05-7756-403c-81f8-2e1a1cad9d15 admin admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:20.181889 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ad6e05-7756-403c-81f8-2e1a1cad9d15 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:13:20.182364 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ad6e05-7756-403c-81f8-2e1a1cad9d15 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:20.185228 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ad6e05-7756-403c-81f8-2e1a1cad9d15 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:20.189639 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 782/1566] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:13:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:13:20.191072 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a14b17-281e-4404-9198-5542533f66db None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:20.203978 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 785/1567] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:13:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:13:20.204857 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a14b17-281e-4404-9198-5542533f66db None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:20.206138 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ad6e05-7756-403c-81f8-2e1a1cad9d15 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:20.209845 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118753d5-2fe0-48b7-ad82-587101a48703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:20.210539 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118753d5-2fe0-48b7-ad82-587101a48703 admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:20.211223 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118753d5-2fe0-48b7-ad82-587101a48703 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:20.211223 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118753d5-2fe0-48b7-ad82-587101a48703 admin admin] PATH_INFO: `/v3/projects/a1842253d6634f1f8b93bfa5be2ddd5f/users/fd36089b3c2141218660ab2458376798/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:20.220348 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ad6e05-7756-403c-81f8-2e1a1cad9d15 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:20.225280 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501bf6aa-dda2-4cd5-ad2c-08e9b46f3846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:20.225962 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501bf6aa-dda2-4cd5-ad2c-08e9b46f3846 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:20.226212 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501bf6aa-dda2-4cd5-ad2c-08e9b46f3846 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:20.226526 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501bf6aa-dda2-4cd5-ad2c-08e9b46f3846 admin admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:20.227186 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501bf6aa-dda2-4cd5-ad2c-08e9b46f3846 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:13:20.227500 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501bf6aa-dda2-4cd5-ad2c-08e9b46f3846 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:20.229699 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501bf6aa-dda2-4cd5-ad2c-08e9b46f3846 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:20.230272 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118753d5-2fe0-48b7-ad82-587101a48703 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1842253d6634f1f8b93bfa5be2ddd5f, user_id=fd36089b3c2141218660ab2458376798, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:20.231979 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: WARNING py.warnings [None req-118753d5-2fe0-48b7-ad82-587101a48703 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 16:13:20.231979 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: warnings.warn(msg) Sep 19 16:13:20.231979 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: Sep 19 16:13:20.234507 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118753d5-2fe0-48b7-ad82-587101a48703 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:20.239903 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:13:20.245015 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 786/1568] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:13:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:13:20.246762 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501bf6aa-dda2-4cd5-ad2c-08e9b46f3846 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:20.256068 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:13:20.259078 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501bf6aa-dda2-4cd5-ad2c-08e9b46f3846 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:20.262176 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 783/1569] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:13:20 2022] PUT /identity/v3/projects/a1842253d6634f1f8b93bfa5be2ddd5f/users/fd36089b3c2141218660ab2458376798/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:13:20.263050 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118753d5-2fe0-48b7-ad82-587101a48703 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:20.326689 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b06cca2-ac53-407d-920a-b17be54b6908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:20.327363 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b06cca2-ac53-407d-920a-b17be54b6908 admin admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:20.327790 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b06cca2-ac53-407d-920a-b17be54b6908 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:20.328074 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b06cca2-ac53-407d-920a-b17be54b6908 admin admin] PATH_INFO: `/v3/projects/8c22de2ade474ef3a075af2ea29cc730/users/95f43fa305e84ebdb268de29df725737/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:20.337211 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118753d5-2fe0-48b7-ad82-587101a48703 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:20.342962 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f257b2-4b57-4f77-b5cb-fef9953f0676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:20.343712 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f257b2-4b57-4f77-b5cb-fef9953f0676 admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:20.344063 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f257b2-4b57-4f77-b5cb-fef9953f0676 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:20.344498 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f257b2-4b57-4f77-b5cb-fef9953f0676 admin admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:20.345642 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f257b2-4b57-4f77-b5cb-fef9953f0676 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:20.346916 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b06cca2-ac53-407d-920a-b17be54b6908 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c22de2ade474ef3a075af2ea29cc730, user_id=95f43fa305e84ebdb268de29df725737, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:20.349042 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f257b2-4b57-4f77-b5cb-fef9953f0676 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:20.351971 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b06cca2-ac53-407d-920a-b17be54b6908 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:20.369196 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 787/1570] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:13:20 2022] PUT /identity/v3/projects/8c22de2ade474ef3a075af2ea29cc730/users/95f43fa305e84ebdb268de29df725737/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:13:20.375971 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae81ba-9249-4898-b5fb-3b3b1c9c96f6 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:20.376276 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae81ba-9249-4898-b5fb-3b3b1c9c96f6 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:20.376533 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae81ba-9249-4898-b5fb-3b3b1c9c96f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:20.386452 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 784/1571] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:13:20 2022] POST /identity/v3/projects => generated 377 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:13:20.387702 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f257b2-4b57-4f77-b5cb-fef9953f0676 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:20.398031 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-01ae81ba-9249-4898-b5fb-3b3b1c9c96f6 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:13:20.399304 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-01ae81ba-9249-4898-b5fb-3b3b1c9c96f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:20.456714 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f257b2-4b57-4f77-b5cb-fef9953f0676 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:20.462397 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d0a2a5-e288-49af-823a-7ea4fe6f77ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:20.463323 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 788/1572] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:13:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:13:20.463866 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d0a2a5-e288-49af-823a-7ea4fe6f77ae admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:20.464257 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ae81ba-9249-4898-b5fb-3b3b1c9c96f6 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:20.464447 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d0a2a5-e288-49af-823a-7ea4fe6f77ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:20.464740 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d0a2a5-e288-49af-823a-7ea4fe6f77ae admin admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:20.465644 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d0a2a5-e288-49af-823a-7ea4fe6f77ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:20.468348 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d0a2a5-e288-49af-823a-7ea4fe6f77ae admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:20.474330 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ae81ba-9249-4898-b5fb-3b3b1c9c96f6 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:20.479232 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89573336-012d-47de-badb-ea0a8e3a8237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:20.480286 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89573336-012d-47de-badb-ea0a8e3a8237 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:20.480698 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89573336-012d-47de-badb-ea0a8e3a8237 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:20.481030 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89573336-012d-47de-badb-ea0a8e3a8237 admin admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:20.481917 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89573336-012d-47de-badb-ea0a8e3a8237 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:13:20.482417 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89573336-012d-47de-badb-ea0a8e3a8237 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:20.484383 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89573336-012d-47de-badb-ea0a8e3a8237 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:20.509851 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 789/1573] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:13:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:13:20.510101 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 785/1574] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:13:20 2022] POST /identity/v3/projects => generated 377 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:13:20.510101 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d0a2a5-e288-49af-823a-7ea4fe6f77ae admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:20.510306 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89573336-012d-47de-badb-ea0a8e3a8237 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:20.520988 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89573336-012d-47de-badb-ea0a8e3a8237 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:20.581307 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d0a2a5-e288-49af-823a-7ea4fe6f77ae admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:20.583466 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b76e08-21d2-4855-85e4-ed9aa279c4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:20.584198 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b76e08-21d2-4855-85e4-ed9aa279c4a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:20.584568 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b76e08-21d2-4855-85e4-ed9aa279c4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:20.584954 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b76e08-21d2-4855-85e4-ed9aa279c4a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:20.585896 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b76e08-21d2-4855-85e4-ed9aa279c4a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:20.586211 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22420387-5679-4d0b-82e3-b544e7887a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:20.586839 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22420387-5679-4d0b-82e3-b544e7887a4f admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:20.587120 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22420387-5679-4d0b-82e3-b544e7887a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:20.587399 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22420387-5679-4d0b-82e3-b544e7887a4f admin admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:20.588107 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22420387-5679-4d0b-82e3-b544e7887a4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:20.588698 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b76e08-21d2-4855-85e4-ed9aa279c4a2 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:20.590001 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22420387-5679-4d0b-82e3-b544e7887a4f admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:20.607510 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:13:20.608866 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:13:20.628906 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 790/1575] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:13:20 2022] POST /identity/v3/projects => generated 361 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:13:20.629336 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 786/1576] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:13:20 2022] POST /identity/v3/projects => generated 391 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:13:20.631426 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334cef45-2193-4add-99ca-2bd2931ff135 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:20.631762 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d864f4-58b8-4e07-bd9a-0976f959028d None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:20.631968 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334cef45-2193-4add-99ca-2bd2931ff135 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:20.632329 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334cef45-2193-4add-99ca-2bd2931ff135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:20.632917 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d864f4-58b8-4e07-bd9a-0976f959028d None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:20.634024 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d864f4-58b8-4e07-bd9a-0976f959028d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:20.662023 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-35d864f4-58b8-4e07-bd9a-0976f959028d None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:13:20.663780 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-35d864f4-58b8-4e07-bd9a-0976f959028d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:20.677199 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-334cef45-2193-4add-99ca-2bd2931ff135 None None] MFA Rules not processed for user `95f43fa305e84ebdb268de29df725737`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:13:20.679225 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-334cef45-2193-4add-99ca-2bd2931ff135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:20.750426 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 791/1577] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:13:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:13:20.751686 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d864f4-58b8-4e07-bd9a-0976f959028d None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:20.765798 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 787/1578] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:13:20 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:13:20.767334 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334cef45-2193-4add-99ca-2bd2931ff135 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:20.768082 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d864f4-58b8-4e07-bd9a-0976f959028d None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:20.777416 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d77f061-f8e7-4a8c-90ee-e06e501e66be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:20.778138 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d77f061-f8e7-4a8c-90ee-e06e501e66be admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:20.778461 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d77f061-f8e7-4a8c-90ee-e06e501e66be admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:20.778809 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d77f061-f8e7-4a8c-90ee-e06e501e66be admin admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:20.779529 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d77f061-f8e7-4a8c-90ee-e06e501e66be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:20.781374 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d77f061-f8e7-4a8c-90ee-e06e501e66be admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:20.782761 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334cef45-2193-4add-99ca-2bd2931ff135 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:20.787965 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741f9763-4f0b-4d05-901c-a25432b511ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:20.788859 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f9763-4f0b-4d05-901c-a25432b511ed admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:20.789272 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f9763-4f0b-4d05-901c-a25432b511ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:20.789929 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f9763-4f0b-4d05-901c-a25432b511ed admin admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:20.791331 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741f9763-4f0b-4d05-901c-a25432b511ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:13:20.791801 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741f9763-4f0b-4d05-901c-a25432b511ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:20.793984 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741f9763-4f0b-4d05-901c-a25432b511ed admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:20.809176 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 792/1579] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:13:20 2022] POST /identity/v3/users => generated 289 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:13:20.809176 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d77f061-f8e7-4a8c-90ee-e06e501e66be admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:20.817087 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 788/1580] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:13:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:13:20.817798 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d77f061-f8e7-4a8c-90ee-e06e501e66be admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:20.819018 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741f9763-4f0b-4d05-901c-a25432b511ed admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:20.828918 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16e410c-0393-487f-a301-6b6aa31d7bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:20.829707 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16e410c-0393-487f-a301-6b6aa31d7bcd admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:20.830219 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16e410c-0393-487f-a301-6b6aa31d7bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:20.830732 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16e410c-0393-487f-a301-6b6aa31d7bcd admin admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:20.831871 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16e410c-0393-487f-a301-6b6aa31d7bcd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:20.834129 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16e410c-0393-487f-a301-6b6aa31d7bcd admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:20.834598 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741f9763-4f0b-4d05-901c-a25432b511ed admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:20.840190 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b08b69-38ac-497c-a8fd-38a05c360a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:20.841100 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b08b69-38ac-497c-a8fd-38a05c360a38 admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:20.841544 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b08b69-38ac-497c-a8fd-38a05c360a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:20.842002 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b08b69-38ac-497c-a8fd-38a05c360a38 admin admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:20.842813 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b08b69-38ac-497c-a8fd-38a05c360a38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:20.845289 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b08b69-38ac-497c-a8fd-38a05c360a38 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:20.858816 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 793/1581] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:13:20 2022] POST /identity/v3/users => generated 289 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:13:20.859768 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16e410c-0393-487f-a301-6b6aa31d7bcd admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:20.870182 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16e410c-0393-487f-a301-6b6aa31d7bcd admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:20.875352 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a27b72-5eab-4fbe-a944-cf3a30fae1f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:20.876057 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a27b72-5eab-4fbe-a944-cf3a30fae1f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:20.876330 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a27b72-5eab-4fbe-a944-cf3a30fae1f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:20.876604 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a27b72-5eab-4fbe-a944-cf3a30fae1f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:20.877244 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a27b72-5eab-4fbe-a944-cf3a30fae1f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:20.879206 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a27b72-5eab-4fbe-a944-cf3a30fae1f0 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:20.882085 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 789/1582] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:13:20 2022] POST /identity/v3/projects => generated 389 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:13:20.882767 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b08b69-38ac-497c-a8fd-38a05c360a38 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:20.908857 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 794/1583] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:13:20 2022] POST /identity/v3/users => generated 296 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:13:20.908857 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a27b72-5eab-4fbe-a944-cf3a30fae1f0 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:20.959567 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b08b69-38ac-497c-a8fd-38a05c360a38 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:20.964344 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5a4875-55ba-49d8-a88d-58852d1e726f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:20.964951 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a4875-55ba-49d8-a88d-58852d1e726f admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:20.965227 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a4875-55ba-49d8-a88d-58852d1e726f admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:20.965496 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a4875-55ba-49d8-a88d-58852d1e726f admin admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:20.966136 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5a4875-55ba-49d8-a88d-58852d1e726f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:20.968135 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5a4875-55ba-49d8-a88d-58852d1e726f admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:20.970088 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a27b72-5eab-4fbe-a944-cf3a30fae1f0 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:20.994986 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 790/1584] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:13:20 2022] POST /identity/v3/users => generated 281 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:13:20.995821 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5a4875-55ba-49d8-a88d-58852d1e726f admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:21.006616 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5a4875-55ba-49d8-a88d-58852d1e726f admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:21.011792 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ec39a4-605c-44a9-844a-15c6339a1fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:21.012404 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec39a4-605c-44a9-844a-15c6339a1fd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:21.012685 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec39a4-605c-44a9-844a-15c6339a1fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:21.012998 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec39a4-605c-44a9-844a-15c6339a1fd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:21.013588 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ec39a4-605c-44a9-844a-15c6339a1fd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:21.015546 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ec39a4-605c-44a9-844a-15c6339a1fd7 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:21.047098 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 791/1585] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:13:20 2022] POST /identity/v3/projects => generated 365 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:13:21.048441 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ec39a4-605c-44a9-844a-15c6339a1fd7 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:21.057103 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c2f941-fd90-43fb-aae2-4c90c8af1918 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:21.058111 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2f941-fd90-43fb-aae2-4c90c8af1918 service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:21.058623 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2f941-fd90-43fb-aae2-4c90c8af1918 service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:21.059102 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2f941-fd90-43fb-aae2-4c90c8af1918 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:21.076943 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c2f941-fd90-43fb-aae2-4c90c8af1918 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:21.079741 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c2f941-fd90-43fb-aae2-4c90c8af1918 service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:21.080192 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:13:21.095245 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 795/1586] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:13:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:13:21.096110 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c2f941-fd90-43fb-aae2-4c90c8af1918 service neutron] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:21.101306 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:13:21.107946 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c2f941-fd90-43fb-aae2-4c90c8af1918 service neutron] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:21.174715 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ec39a4-605c-44a9-844a-15c6339a1fd7 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:21.181103 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d6bd44-cf0f-48fd-a49a-5971fdfd4092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:21.181766 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d6bd44-cf0f-48fd-a49a-5971fdfd4092 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:21.182048 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d6bd44-cf0f-48fd-a49a-5971fdfd4092 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:21.182323 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d6bd44-cf0f-48fd-a49a-5971fdfd4092 admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:21.183018 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d6bd44-cf0f-48fd-a49a-5971fdfd4092 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:21.184864 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d6bd44-cf0f-48fd-a49a-5971fdfd4092 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:21.196105 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 792/1587] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:13:21 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:13:21.196728 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d6bd44-cf0f-48fd-a49a-5971fdfd4092 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:21.203975 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605d5372-d275-466c-8b78-e74c36d9bd64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:21.204994 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605d5372-d275-466c-8b78-e74c36d9bd64 admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:21.205454 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605d5372-d275-466c-8b78-e74c36d9bd64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:21.205935 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605d5372-d275-466c-8b78-e74c36d9bd64 admin admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:21.207179 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605d5372-d275-466c-8b78-e74c36d9bd64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:21.210896 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605d5372-d275-466c-8b78-e74c36d9bd64 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:21.212333 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d6bd44-cf0f-48fd-a49a-5971fdfd4092 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:21.218040 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad6f11f-4b60-4341-a425-a2b24204365c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:21.218839 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad6f11f-4b60-4341-a425-a2b24204365c admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:21.219137 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad6f11f-4b60-4341-a425-a2b24204365c admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:21.219436 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad6f11f-4b60-4341-a425-a2b24204365c admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:21.220035 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad6f11f-4b60-4341-a425-a2b24204365c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:21.222161 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad6f11f-4b60-4341-a425-a2b24204365c admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:21.236639 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 793/1588] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:13:21 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:13:21.237766 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad6f11f-4b60-4341-a425-a2b24204365c admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:21.249518 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad6f11f-4b60-4341-a425-a2b24204365c admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:21.255157 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac32c6b-bdf9-4bc6-9ca6-0ed3b121fbdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:21.256432 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac32c6b-bdf9-4bc6-9ca6-0ed3b121fbdb admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:21.256814 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac32c6b-bdf9-4bc6-9ca6-0ed3b121fbdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:21.257005 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac32c6b-bdf9-4bc6-9ca6-0ed3b121fbdb admin admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:21.257618 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac32c6b-bdf9-4bc6-9ca6-0ed3b121fbdb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:21.260114 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac32c6b-bdf9-4bc6-9ca6-0ed3b121fbdb admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:21.262756 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 796/1589] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:13:21 2022] POST /identity/v3/projects => generated 381 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:13:21.264004 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605d5372-d275-466c-8b78-e74c36d9bd64 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:21.307830 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 794/1590] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:13:21 2022] POST /identity/v3/users => generated 295 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:13:21.307830 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac32c6b-bdf9-4bc6-9ca6-0ed3b121fbdb admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:21.324600 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac32c6b-bdf9-4bc6-9ca6-0ed3b121fbdb admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:21.354459 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605d5372-d275-466c-8b78-e74c36d9bd64 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:21.362865 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9075e8-41be-40b1-a95c-bb69ea5739f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:21.363851 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9075e8-41be-40b1-a95c-bb69ea5739f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:21.364341 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9075e8-41be-40b1-a95c-bb69ea5739f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:21.364823 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9075e8-41be-40b1-a95c-bb69ea5739f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:21.366001 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9075e8-41be-40b1-a95c-bb69ea5739f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:21.369873 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9075e8-41be-40b1-a95c-bb69ea5739f6 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:21.389575 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 797/1591] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:13:21 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:13:21.391004 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9075e8-41be-40b1-a95c-bb69ea5739f6 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:21.408780 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9075e8-41be-40b1-a95c-bb69ea5739f6 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:21.417424 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcedf32-ce13-491e-9391-d02065aa9dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:21.418186 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcedf32-ce13-491e-9391-d02065aa9dee admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:21.418618 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcedf32-ce13-491e-9391-d02065aa9dee admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:21.419024 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcedf32-ce13-491e-9391-d02065aa9dee admin admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:21.419817 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcedf32-ce13-491e-9391-d02065aa9dee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:21.422243 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcedf32-ce13-491e-9391-d02065aa9dee admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:21.447478 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e82923c-a11e-4546-a0e6-7f5cb9b4079a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:21.448435 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e82923c-a11e-4546-a0e6-7f5cb9b4079a admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:21.448727 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e82923c-a11e-4546-a0e6-7f5cb9b4079a admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:21.448999 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e82923c-a11e-4546-a0e6-7f5cb9b4079a admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:21.449663 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e82923c-a11e-4546-a0e6-7f5cb9b4079a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:21.451671 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e82923c-a11e-4546-a0e6-7f5cb9b4079a admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:21.455023 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 798/1592] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:13:21 2022] POST /identity/v3/users => generated 289 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:13:21.455770 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcedf32-ce13-491e-9391-d02065aa9dee admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:21.468830 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 795/1593] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:13:21 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:13:21.468830 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e82923c-a11e-4546-a0e6-7f5cb9b4079a admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:21.473861 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcedf32-ce13-491e-9391-d02065aa9dee admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:21.477268 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e82923c-a11e-4546-a0e6-7f5cb9b4079a admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:21.481499 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535a82c0-4741-4513-b1d7-98d106e29be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:21.482719 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535a82c0-4741-4513-b1d7-98d106e29be1 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:21.482968 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535a82c0-4741-4513-b1d7-98d106e29be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:21.483365 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535a82c0-4741-4513-b1d7-98d106e29be1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:21.483541 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4408f5dd-3a87-4b1d-bc0b-3356255c22cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:21.484775 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535a82c0-4741-4513-b1d7-98d106e29be1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:21.485027 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4408f5dd-3a87-4b1d-bc0b-3356255c22cf admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:21.485027 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4408f5dd-3a87-4b1d-bc0b-3356255c22cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:21.485027 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4408f5dd-3a87-4b1d-bc0b-3356255c22cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:21.485805 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4408f5dd-3a87-4b1d-bc0b-3356255c22cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:21.486078 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535a82c0-4741-4513-b1d7-98d106e29be1 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:21.487904 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4408f5dd-3a87-4b1d-bc0b-3356255c22cf admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:21.501292 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 796/1594] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:13:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:13:21.502416 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4408f5dd-3a87-4b1d-bc0b-3356255c22cf admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:21.507796 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 799/1595] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:13:21 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:13:21.509597 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535a82c0-4741-4513-b1d7-98d106e29be1 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:21.515040 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4408f5dd-3a87-4b1d-bc0b-3356255c22cf admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:21.519991 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3321ddff-64ca-416e-bc6b-290b7d285c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:21.520699 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3321ddff-64ca-416e-bc6b-290b7d285c85 admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:21.521102 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3321ddff-64ca-416e-bc6b-290b7d285c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:21.521421 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3321ddff-64ca-416e-bc6b-290b7d285c85 admin admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:21.522161 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3321ddff-64ca-416e-bc6b-290b7d285c85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:21.524197 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3321ddff-64ca-416e-bc6b-290b7d285c85 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:21.528455 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535a82c0-4741-4513-b1d7-98d106e29be1 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:21.535393 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27df4d77-533d-462f-b698-4330467cf67a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:21.536367 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27df4d77-533d-462f-b698-4330467cf67a admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:21.536983 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27df4d77-533d-462f-b698-4330467cf67a admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:21.537357 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27df4d77-533d-462f-b698-4330467cf67a admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:21.538458 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27df4d77-533d-462f-b698-4330467cf67a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:21.541801 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27df4d77-533d-462f-b698-4330467cf67a admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:21.551722 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 797/1596] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:13:21 2022] POST /identity/v3/users => generated 291 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:13:21.557494 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3321ddff-64ca-416e-bc6b-290b7d285c85 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:21.560248 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 800/1597] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:13:21 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:13:21.561748 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27df4d77-533d-462f-b698-4330467cf67a admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:21.578001 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27df4d77-533d-462f-b698-4330467cf67a admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:21.579392 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3321ddff-64ca-416e-bc6b-290b7d285c85 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:21.584146 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d5760f-7a83-4c0a-b8a1-d67ff0acba88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:21.584894 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d5760f-7a83-4c0a-b8a1-d67ff0acba88 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:21.585373 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d5760f-7a83-4c0a-b8a1-d67ff0acba88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:21.585821 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d5760f-7a83-4c0a-b8a1-d67ff0acba88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:21.586642 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d5760f-7a83-4c0a-b8a1-d67ff0acba88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:21.588786 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253f086f-65be-467e-9a6f-3994ecb7369c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:21.589477 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d5760f-7a83-4c0a-b8a1-d67ff0acba88 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:21.589667 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f086f-65be-467e-9a6f-3994ecb7369c admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:21.590124 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f086f-65be-467e-9a6f-3994ecb7369c admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:21.590695 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f086f-65be-467e-9a6f-3994ecb7369c admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:21.591764 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253f086f-65be-467e-9a6f-3994ecb7369c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:21.595008 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253f086f-65be-467e-9a6f-3994ecb7369c admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:21.615771 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 801/1598] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:13:21 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:13:21.615771 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d5760f-7a83-4c0a-b8a1-d67ff0acba88 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:21.616215 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 798/1599] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:13:21 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:13:21.616215 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253f086f-65be-467e-9a6f-3994ecb7369c admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:21.622620 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253f086f-65be-467e-9a6f-3994ecb7369c admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:21.622878 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d5760f-7a83-4c0a-b8a1-d67ff0acba88 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:21.629233 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b209fb0-500a-444b-8eff-663bc082a76b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:21.629815 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa20bc2-86a3-4c0b-913b-d13a53c1e3f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:21.630475 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b209fb0-500a-444b-8eff-663bc082a76b admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:21.632729 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b209fb0-500a-444b-8eff-663bc082a76b admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:21.632908 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa20bc2-86a3-4c0b-913b-d13a53c1e3f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:21.633200 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b209fb0-500a-444b-8eff-663bc082a76b admin admin] PATH_INFO: `/v3/projects/23c705bc001e4228bdfcdfa916a368e9/users/754c1b5cb47746cf93c52d6dc0b8fa01/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:21.633430 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa20bc2-86a3-4c0b-913b-d13a53c1e3f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:21.633430 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa20bc2-86a3-4c0b-913b-d13a53c1e3f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:21.633430 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa20bc2-86a3-4c0b-913b-d13a53c1e3f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:21.636676 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa20bc2-86a3-4c0b-913b-d13a53c1e3f0 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:21.655719 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b209fb0-500a-444b-8eff-663bc082a76b admin admin] RBAC: Authorizing `identity:create_grant(project_id=23c705bc001e4228bdfcdfa916a368e9, user_id=754c1b5cb47746cf93c52d6dc0b8fa01, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:21.656224 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 802/1600] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:13:21 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:13:21.658780 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b209fb0-500a-444b-8eff-663bc082a76b admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:21.662352 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa20bc2-86a3-4c0b-913b-d13a53c1e3f0 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:21.676162 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa20bc2-86a3-4c0b-913b-d13a53c1e3f0 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:21.682874 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 799/1601] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:13:21 2022] PUT /identity/v3/projects/23c705bc001e4228bdfcdfa916a368e9/users/754c1b5cb47746cf93c52d6dc0b8fa01/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:13:21.684288 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b209fb0-500a-444b-8eff-663bc082a76b admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:21.705069 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:13:21.711035 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:13:21.776455 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b209fb0-500a-444b-8eff-663bc082a76b admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:21.786182 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65da42f2-128d-47b9-be30-42cdfd7e468f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:21.787138 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65da42f2-128d-47b9-be30-42cdfd7e468f admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:21.787595 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65da42f2-128d-47b9-be30-42cdfd7e468f admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:21.787877 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65da42f2-128d-47b9-be30-42cdfd7e468f admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:21.789104 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65da42f2-128d-47b9-be30-42cdfd7e468f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:21.791406 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65da42f2-128d-47b9-be30-42cdfd7e468f admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:21.798443 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aa23ed-907d-4fc5-83ad-feec5e24c530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:21.802406 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aa23ed-907d-4fc5-83ad-feec5e24c530 admin admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:21.802909 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aa23ed-907d-4fc5-83ad-feec5e24c530 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:21.804122 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aa23ed-907d-4fc5-83ad-feec5e24c530 admin admin] PATH_INFO: `/v3/projects/34baf55057ee42ffbd099de2d65544a1/users/428a95af27d2468bb39391ca3eefea40/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:21.804734 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 800/1602] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:13:21 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:13:21.805792 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65da42f2-128d-47b9-be30-42cdfd7e468f admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:21.817734 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65da42f2-128d-47b9-be30-42cdfd7e468f admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:21.823927 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0c81f1-f9d3-434e-aeed-ea4862a8ff7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:21.824729 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0c81f1-f9d3-434e-aeed-ea4862a8ff7e admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:21.825212 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0c81f1-f9d3-434e-aeed-ea4862a8ff7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:21.825393 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0c81f1-f9d3-434e-aeed-ea4862a8ff7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:21.825986 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0c81f1-f9d3-434e-aeed-ea4862a8ff7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:21.828125 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0c81f1-f9d3-434e-aeed-ea4862a8ff7e admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:21.831388 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aa23ed-907d-4fc5-83ad-feec5e24c530 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34baf55057ee42ffbd099de2d65544a1, user_id=428a95af27d2468bb39391ca3eefea40, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:21.837270 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aa23ed-907d-4fc5-83ad-feec5e24c530 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:21.843947 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 801/1603] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:13:21 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:13:21.845217 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0c81f1-f9d3-434e-aeed-ea4862a8ff7e admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:21.859036 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0c81f1-f9d3-434e-aeed-ea4862a8ff7e admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:21.860516 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 803/1604] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:13:21 2022] PUT /identity/v3/projects/34baf55057ee42ffbd099de2d65544a1/users/428a95af27d2468bb39391ca3eefea40/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:13:21.865779 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aa23ed-907d-4fc5-83ad-feec5e24c530 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:21.934358 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c81eca-21d8-4c28-a84c-e0ae20564b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:21.935457 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c81eca-21d8-4c28-a84c-e0ae20564b8c admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:21.935457 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c81eca-21d8-4c28-a84c-e0ae20564b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:21.935980 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c81eca-21d8-4c28-a84c-e0ae20564b8c admin admin] PATH_INFO: `/v3/projects/0e35510b626e46aaae189f31b27ed663/users/8cd9c727c0b444cb83497ea4f78e24d0/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:21.957910 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c81eca-21d8-4c28-a84c-e0ae20564b8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e35510b626e46aaae189f31b27ed663, user_id=8cd9c727c0b444cb83497ea4f78e24d0, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:21.960659 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c81eca-21d8-4c28-a84c-e0ae20564b8c admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:21.970665 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aa23ed-907d-4fc5-83ad-feec5e24c530 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:21.976910 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535929fa-35b4-4500-94d9-a87786f4a278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:21.977597 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535929fa-35b4-4500-94d9-a87786f4a278 admin admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:21.977911 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535929fa-35b4-4500-94d9-a87786f4a278 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:21.978802 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535929fa-35b4-4500-94d9-a87786f4a278 admin admin] PATH_INFO: `/v3/projects/e1468393e72147489fcf7e30df416868/users/a321283a551b47de86255f93830017e6/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:21.985662 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 802/1605] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:13:21 2022] PUT /identity/v3/projects/0e35510b626e46aaae189f31b27ed663/users/8cd9c727c0b444cb83497ea4f78e24d0/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:13:21.987064 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c81eca-21d8-4c28-a84c-e0ae20564b8c admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:22.000390 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535929fa-35b4-4500-94d9-a87786f4a278 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1468393e72147489fcf7e30df416868, user_id=a321283a551b47de86255f93830017e6, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:22.004656 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535929fa-35b4-4500-94d9-a87786f4a278 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:22.027672 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 804/1606] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:13:21 2022] PUT /identity/v3/projects/e1468393e72147489fcf7e30df416868/users/a321283a551b47de86255f93830017e6/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:13:22.030855 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15371209-5dbd-4057-9c19-2eb91b269cdf None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:22.032220 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15371209-5dbd-4057-9c19-2eb91b269cdf None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:22.033157 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15371209-5dbd-4057-9c19-2eb91b269cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:22.070408 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-15371209-5dbd-4057-9c19-2eb91b269cdf None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:13:22.073238 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-15371209-5dbd-4057-9c19-2eb91b269cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:22.083239 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c81eca-21d8-4c28-a84c-e0ae20564b8c admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:22.132003 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:13:22.186118 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e26663-ff9b-4341-8c2f-acfcab940688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:22.200241 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e26663-ff9b-4341-8c2f-acfcab940688 admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:22.200698 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e26663-ff9b-4341-8c2f-acfcab940688 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:22.201161 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e26663-ff9b-4341-8c2f-acfcab940688 admin admin] PATH_INFO: `/v3/projects/2dc0c57ce73a415990562e3113a7b316/users/597611fe2b3e44d5b9f24ae1158e49d3/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:22.207656 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 805/1607] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:13:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:13:22.207656 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15371209-5dbd-4057-9c19-2eb91b269cdf None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:22.242975 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15371209-5dbd-4057-9c19-2eb91b269cdf None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:22.245586 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e26663-ff9b-4341-8c2f-acfcab940688 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dc0c57ce73a415990562e3113a7b316, user_id=597611fe2b3e44d5b9f24ae1158e49d3, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:22.251541 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3e5141-a5f0-4092-be23-01755b708d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:22.251780 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e26663-ff9b-4341-8c2f-acfcab940688 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:22.265619 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3e5141-a5f0-4092-be23-01755b708d02 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:22.266721 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3e5141-a5f0-4092-be23-01755b708d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:22.267962 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3e5141-a5f0-4092-be23-01755b708d02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:22.268853 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3e5141-a5f0-4092-be23-01755b708d02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:22.270934 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3e5141-a5f0-4092-be23-01755b708d02 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:22.291585 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 806/1608] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:13:22 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:13:22.291585 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3e5141-a5f0-4092-be23-01755b708d02 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:22.291903 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 803/1609] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:13:21 2022] PUT /identity/v3/projects/2dc0c57ce73a415990562e3113a7b316/users/597611fe2b3e44d5b9f24ae1158e49d3/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:13:22.295686 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4368891e-0ce9-4ff8-9040-77dea4d7c85f None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:22.296169 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4368891e-0ce9-4ff8-9040-77dea4d7c85f None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:22.296610 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4368891e-0ce9-4ff8-9040-77dea4d7c85f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:22.312024 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3e5141-a5f0-4092-be23-01755b708d02 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:22.335080 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:13:22.342969 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-4368891e-0ce9-4ff8-9040-77dea4d7c85f None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:13:22.347152 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-4368891e-0ce9-4ff8-9040-77dea4d7c85f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:22.428514 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b3f6a4-4487-44db-88b5-b8abf9e75c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:22.429596 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b3f6a4-4487-44db-88b5-b8abf9e75c5f admin admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:22.430052 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b3f6a4-4487-44db-88b5-b8abf9e75c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:22.430543 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b3f6a4-4487-44db-88b5-b8abf9e75c5f admin admin] PATH_INFO: `/v3/projects/344bf851ff7d471fba47bde9b6f7e219/users/bd190d8e252342ca9232f7a0c966c715/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:22.463559 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b3f6a4-4487-44db-88b5-b8abf9e75c5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=344bf851ff7d471fba47bde9b6f7e219, user_id=bd190d8e252342ca9232f7a0c966c715, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:22.470533 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 804/1610] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:13:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:13:22.471563 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b3f6a4-4487-44db-88b5-b8abf9e75c5f admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:22.474435 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fcf815-def2-4b62-ae5a-467c47acb70b None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:22.474932 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fcf815-def2-4b62-ae5a-467c47acb70b None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:22.475114 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fcf815-def2-4b62-ae5a-467c47acb70b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:22.499335 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-f9fcf815-def2-4b62-ae5a-467c47acb70b None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:13:22.500771 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-f9fcf815-def2-4b62-ae5a-467c47acb70b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:22.506432 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 807/1611] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:13:22 2022] PUT /identity/v3/projects/344bf851ff7d471fba47bde9b6f7e219/users/bd190d8e252342ca9232f7a0c966c715/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:13:22.506432 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b3f6a4-4487-44db-88b5-b8abf9e75c5f admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:22.603627 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 805/1612] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:13:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:13:22.604751 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fcf815-def2-4b62-ae5a-467c47acb70b None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:22.641324 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b3f6a4-4487-44db-88b5-b8abf9e75c5f admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:22.648950 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c96b90e-c3b6-4aa0-b807-b81a73988c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:22.650507 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c96b90e-c3b6-4aa0-b807-b81a73988c6b admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:22.651597 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c96b90e-c3b6-4aa0-b807-b81a73988c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:22.652050 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c96b90e-c3b6-4aa0-b807-b81a73988c6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:22.653186 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c96b90e-c3b6-4aa0-b807-b81a73988c6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:22.656227 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c96b90e-c3b6-4aa0-b807-b81a73988c6b admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:22.682116 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 808/1613] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:13:22 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:13:22.682116 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c96b90e-c3b6-4aa0-b807-b81a73988c6b admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:22.702307 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c96b90e-c3b6-4aa0-b807-b81a73988c6b admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:22.711904 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a6393e-ebab-4ec4-9ba2-896fd659384a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:22.713550 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a6393e-ebab-4ec4-9ba2-896fd659384a admin admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:22.714656 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a6393e-ebab-4ec4-9ba2-896fd659384a admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:22.715135 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a6393e-ebab-4ec4-9ba2-896fd659384a admin admin] PATH_INFO: `/v3/projects/0350ad6e81224f0b801874c75828e039/users/6e24f66ae3704054a89cc44eaf9663ff/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:22.733139 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fcf815-def2-4b62-ae5a-467c47acb70b None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:22.742765 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0ce035-261c-4082-80f6-6eb3c885933c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:22.743909 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a6393e-ebab-4ec4-9ba2-896fd659384a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0350ad6e81224f0b801874c75828e039, user_id=6e24f66ae3704054a89cc44eaf9663ff, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:22.744401 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0ce035-261c-4082-80f6-6eb3c885933c service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:22.745821 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0ce035-261c-4082-80f6-6eb3c885933c service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:22.746477 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0ce035-261c-4082-80f6-6eb3c885933c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:22.749694 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a6393e-ebab-4ec4-9ba2-896fd659384a admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:22.777672 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0ce035-261c-4082-80f6-6eb3c885933c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:22.779616 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0ce035-261c-4082-80f6-6eb3c885933c service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:22.797690 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 809/1614] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:13:22 2022] PUT /identity/v3/projects/0350ad6e81224f0b801874c75828e039/users/6e24f66ae3704054a89cc44eaf9663ff/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:13:22.797690 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5009860d-6a3c-48c5-934e-d00f7a1de952 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:22.797690 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5009860d-6a3c-48c5-934e-d00f7a1de952 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:22.798188 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:13:22.798288 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5009860d-6a3c-48c5-934e-d00f7a1de952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:22.804787 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 806/1615] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:13:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:13:22.807296 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0ce035-261c-4082-80f6-6eb3c885933c service neutron] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:22.832908 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0ce035-261c-4082-80f6-6eb3c885933c service neutron] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:22.874270 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-5009860d-6a3c-48c5-934e-d00f7a1de952 None None] MFA Rules not processed for user `597611fe2b3e44d5b9f24ae1158e49d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:13:22.877664 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-5009860d-6a3c-48c5-934e-d00f7a1de952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:22.990151 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8735ba16-bfbd-4d6e-b827-3564b5b8c685 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:22.991830 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8735ba16-bfbd-4d6e-b827-3564b5b8c685 service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:22.992294 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8735ba16-bfbd-4d6e-b827-3564b5b8c685 service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:22.993097 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8735ba16-bfbd-4d6e-b827-3564b5b8c685 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:23.050978 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 810/1616] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:13:22 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:13:23.052297 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d68303f-7ae9-44e9-8608-9b636b3bc1b2 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:23.052565 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d68303f-7ae9-44e9-8608-9b636b3bc1b2 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:23.052843 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d68303f-7ae9-44e9-8608-9b636b3bc1b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:23.075645 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:13:23.097713 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-2d68303f-7ae9-44e9-8608-9b636b3bc1b2 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:13:23.099111 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-2d68303f-7ae9-44e9-8608-9b636b3bc1b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:23.214488 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8735ba16-bfbd-4d6e-b827-3564b5b8c685 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:23.218729 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8735ba16-bfbd-4d6e-b827-3564b5b8c685 service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:23.246794 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 811/1617] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:13:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:13:23.247696 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d68303f-7ae9-44e9-8608-9b636b3bc1b2 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:23.259111 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 807/1618] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:13:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:13:23.263978 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d68303f-7ae9-44e9-8608-9b636b3bc1b2 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:23.264186 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8735ba16-bfbd-4d6e-b827-3564b5b8c685 service neutron] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:23.270841 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4e0140-bce7-473c-a059-990b59d3bab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:23.271570 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4e0140-bce7-473c-a059-990b59d3bab6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:23.271855 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4e0140-bce7-473c-a059-990b59d3bab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:23.272138 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4e0140-bce7-473c-a059-990b59d3bab6 admin admin] PATH_INFO: `/v3/projects/e1468393e72147489fcf7e30df416868/users/a321283a551b47de86255f93830017e6/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:23.274842 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4e0140-bce7-473c-a059-990b59d3bab6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1468393e72147489fcf7e30df416868, user_id=a321283a551b47de86255f93830017e6, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:23.277997 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4e0140-bce7-473c-a059-990b59d3bab6 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:23.288654 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8735ba16-bfbd-4d6e-b827-3564b5b8c685 service neutron] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:23.301392 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 812/1619] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:13:23 2022] PUT /identity/v3/projects/e1468393e72147489fcf7e30df416868/users/a321283a551b47de86255f93830017e6/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:13:23.317934 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5707b526-febc-4cc1-beeb-7f3cad596122 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:23.317934 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5707b526-febc-4cc1-beeb-7f3cad596122 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:23.317934 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5707b526-febc-4cc1-beeb-7f3cad596122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:23.387203 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-5707b526-febc-4cc1-beeb-7f3cad596122 None None] MFA Rules not processed for user `6e24f66ae3704054a89cc44eaf9663ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:13:23.389149 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-5707b526-febc-4cc1-beeb-7f3cad596122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:23.408119 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b5438-cfd0-4b4f-a59e-cb14741d8cb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:23.409216 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b5438-cfd0-4b4f-a59e-cb14741d8cb6 service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:23.409529 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b5438-cfd0-4b4f-a59e-cb14741d8cb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:23.409842 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b5438-cfd0-4b4f-a59e-cb14741d8cb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:23.517970 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145b5438-cfd0-4b4f-a59e-cb14741d8cb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:23.520806 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 813/1620] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:13:23 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:13:23.521087 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145b5438-cfd0-4b4f-a59e-cb14741d8cb6 service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:23.521290 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5707b526-febc-4cc1-beeb-7f3cad596122 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:23.531168 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:13:23.612170 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 808/1621] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:13:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:13:23.612170 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b5438-cfd0-4b4f-a59e-cb14741d8cb6 service neutron] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:23.612170 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b5438-cfd0-4b4f-a59e-cb14741d8cb6 service neutron] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:23.612170 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a3fad7-29c6-48ec-a311-6d0601709060 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:23.613293 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a3fad7-29c6-48ec-a311-6d0601709060 service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:23.613293 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a3fad7-29c6-48ec-a311-6d0601709060 service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:23.613293 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a3fad7-29c6-48ec-a311-6d0601709060 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:23.613803 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5707b526-febc-4cc1-beeb-7f3cad596122 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:23.613803 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fb0bac-0ef7-475e-affd-1535c039b1ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:23.613803 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fb0bac-0ef7-475e-affd-1535c039b1ab admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:23.614228 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a3fad7-29c6-48ec-a311-6d0601709060 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:23.616709 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fb0bac-0ef7-475e-affd-1535c039b1ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:23.616709 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fb0bac-0ef7-475e-affd-1535c039b1ab admin admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:23.616709 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fb0bac-0ef7-475e-affd-1535c039b1ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:23.619339 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a3fad7-29c6-48ec-a311-6d0601709060 service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:23.633457 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fb0bac-0ef7-475e-affd-1535c039b1ab admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:23.676229 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 809/1622] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:13:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:13:23.684147 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3626ecd-94a5-4b78-aa42-94a38a3c2ac0 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:23.684581 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3626ecd-94a5-4b78-aa42-94a38a3c2ac0 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:23.684961 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3626ecd-94a5-4b78-aa42-94a38a3c2ac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:23.736881 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 814/1623] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:13:23 2022] POST /identity/v3/projects => generated 363 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:13:23.738574 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fb0bac-0ef7-475e-affd-1535c039b1ab admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:23.766610 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-f3626ecd-94a5-4b78-aa42-94a38a3c2ac0 None None] MFA Rules not processed for user `a321283a551b47de86255f93830017e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:13:23.769098 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-f3626ecd-94a5-4b78-aa42-94a38a3c2ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:23.888017 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:13:23.896540 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fb0bac-0ef7-475e-affd-1535c039b1ab admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:23.916589 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955bbb81-2b43-42ba-a3f9-585615494f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:23.917399 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955bbb81-2b43-42ba-a3f9-585615494f35 admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:23.917677 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955bbb81-2b43-42ba-a3f9-585615494f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:23.926311 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955bbb81-2b43-42ba-a3f9-585615494f35 admin admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:23.927311 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955bbb81-2b43-42ba-a3f9-585615494f35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:23.931841 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955bbb81-2b43-42ba-a3f9-585615494f35 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:23.976475 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 815/1624] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:13:23 2022] POST /identity/v3/projects => generated 379 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:13:23.977537 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955bbb81-2b43-42ba-a3f9-585615494f35 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:23.981069 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 810/1625] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:13:23 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:13:23.987022 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3626ecd-94a5-4b78-aa42-94a38a3c2ac0 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:24.030151 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3626ecd-94a5-4b78-aa42-94a38a3c2ac0 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:24.098726 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955bbb81-2b43-42ba-a3f9-585615494f35 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:24.106943 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e43709-9f67-46d7-a845-07f7613b233d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:24.108452 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e43709-9f67-46d7-a845-07f7613b233d admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:24.108941 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e43709-9f67-46d7-a845-07f7613b233d admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:24.109421 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e43709-9f67-46d7-a845-07f7613b233d admin admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:24.111811 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e43709-9f67-46d7-a845-07f7613b233d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:24.115188 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e43709-9f67-46d7-a845-07f7613b233d admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:24.138133 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aea037-70b9-4f72-881e-ef59fa0a5f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:24.139463 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aea037-70b9-4f72-881e-ef59fa0a5f7d admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:24.139751 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aea037-70b9-4f72-881e-ef59fa0a5f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:24.140183 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aea037-70b9-4f72-881e-ef59fa0a5f7d admin admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:24.140897 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aea037-70b9-4f72-881e-ef59fa0a5f7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:24.143184 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aea037-70b9-4f72-881e-ef59fa0a5f7d admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:24.157044 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 816/1626] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:13:23 2022] POST /identity/v3/users => generated 288 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:13:24.159272 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e43709-9f67-46d7-a845-07f7613b233d admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:24.188647 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 811/1627] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:13:23 2022] POST /identity/v3/users => generated 296 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:13:24.189475 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aea037-70b9-4f72-881e-ef59fa0a5f7d admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:24.225438 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aea037-70b9-4f72-881e-ef59fa0a5f7d admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:24.236562 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7ea7b4-c9c2-4fd3-8a9e-240fdfee4807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:24.238433 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ea7b4-c9c2-4fd3-8a9e-240fdfee4807 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:24.238732 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ea7b4-c9c2-4fd3-8a9e-240fdfee4807 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:24.238991 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ea7b4-c9c2-4fd3-8a9e-240fdfee4807 admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:24.239669 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7ea7b4-c9c2-4fd3-8a9e-240fdfee4807 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:24.244685 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7ea7b4-c9c2-4fd3-8a9e-240fdfee4807 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:24.283145 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 812/1628] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:13:24 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:13:24.283145 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7ea7b4-c9c2-4fd3-8a9e-240fdfee4807 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:24.287563 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e43709-9f67-46d7-a845-07f7613b233d admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:24.298021 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdc4390-d877-4849-82d2-10a7a9bf9128 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:24.299987 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdc4390-d877-4849-82d2-10a7a9bf9128 service cinder] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:24.300295 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdc4390-d877-4849-82d2-10a7a9bf9128 service cinder] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:24.300574 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdc4390-d877-4849-82d2-10a7a9bf9128 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:24.316814 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7ea7b4-c9c2-4fd3-8a9e-240fdfee4807 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:24.329279 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d669f866-48b6-4925-a3ab-886ed2ee9ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:24.330899 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d669f866-48b6-4925-a3ab-886ed2ee9ef9 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:24.331182 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d669f866-48b6-4925-a3ab-886ed2ee9ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:24.331460 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d669f866-48b6-4925-a3ab-886ed2ee9ef9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:24.332372 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d669f866-48b6-4925-a3ab-886ed2ee9ef9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:24.337317 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d669f866-48b6-4925-a3ab-886ed2ee9ef9 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:24.371614 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 813/1629] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:13:24 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:13:24.371614 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d669f866-48b6-4925-a3ab-886ed2ee9ef9 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:24.396419 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d669f866-48b6-4925-a3ab-886ed2ee9ef9 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:24.402266 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca6b53a-1e18-462f-8afa-bca4c066d08e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:24.402991 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca6b53a-1e18-462f-8afa-bca4c066d08e admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:24.403333 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca6b53a-1e18-462f-8afa-bca4c066d08e admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:24.403612 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca6b53a-1e18-462f-8afa-bca4c066d08e admin admin] PATH_INFO: `/v3/projects/71fe5565b854460191eaa4fd12286bea/users/a8b0d4bdfaf345deadd2a4e28c8da2a1/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:24.433281 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca6b53a-1e18-462f-8afa-bca4c066d08e admin admin] RBAC: Authorizing `identity:create_grant(project_id=71fe5565b854460191eaa4fd12286bea, user_id=a8b0d4bdfaf345deadd2a4e28c8da2a1, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:24.438303 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca6b53a-1e18-462f-8afa-bca4c066d08e admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:24.468211 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 814/1630] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:13:24 2022] PUT /identity/v3/projects/71fe5565b854460191eaa4fd12286bea/users/a8b0d4bdfaf345deadd2a4e28c8da2a1/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:13:24.469428 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca6b53a-1e18-462f-8afa-bca4c066d08e admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:24.480245 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdc4390-d877-4849-82d2-10a7a9bf9128 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:24.483529 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdc4390-d877-4849-82d2-10a7a9bf9128 service cinder] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:24.519981 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 817/1631] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:13:24 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:13:24.523353 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdc4390-d877-4849-82d2-10a7a9bf9128 service cinder] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:24.527545 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:13:24.550625 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdc4390-d877-4849-82d2-10a7a9bf9128 service cinder] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:24.582216 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:13:24.660489 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca6b53a-1e18-462f-8afa-bca4c066d08e admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:24.673422 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0146ed4-8dd8-4daa-8524-d5855e9b2a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:24.674445 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52273fa8-78a2-44d2-b639-7d6d69fc1923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:24.674765 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0146ed4-8dd8-4daa-8524-d5855e9b2a89 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:24.675052 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0146ed4-8dd8-4daa-8524-d5855e9b2a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:24.675329 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0146ed4-8dd8-4daa-8524-d5855e9b2a89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:24.676004 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0146ed4-8dd8-4daa-8524-d5855e9b2a89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:24.676479 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52273fa8-78a2-44d2-b639-7d6d69fc1923 admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:24.676769 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52273fa8-78a2-44d2-b639-7d6d69fc1923 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:24.678086 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0146ed4-8dd8-4daa-8524-d5855e9b2a89 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:24.679244 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52273fa8-78a2-44d2-b639-7d6d69fc1923 admin admin] PATH_INFO: `/v3/projects/978518a226d6471cba63f0cef4c857f6/users/2a35fbc2727346b5b0e19158119a753a/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:24.702083 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 818/1632] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:13:24 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:13:24.707151 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0146ed4-8dd8-4daa-8524-d5855e9b2a89 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:24.714567 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52273fa8-78a2-44d2-b639-7d6d69fc1923 admin admin] RBAC: Authorizing `identity:create_grant(project_id=978518a226d6471cba63f0cef4c857f6, user_id=2a35fbc2727346b5b0e19158119a753a, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:24.717697 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52273fa8-78a2-44d2-b639-7d6d69fc1923 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:24.736697 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0146ed4-8dd8-4daa-8524-d5855e9b2a89 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:24.757523 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 815/1633] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:13:24 2022] PUT /identity/v3/projects/978518a226d6471cba63f0cef4c857f6/users/2a35fbc2727346b5b0e19158119a753a/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:13:24.757523 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52273fa8-78a2-44d2-b639-7d6d69fc1923 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:24.893781 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd22511-edd9-468c-b2dd-293182453403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:24.907237 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd22511-edd9-468c-b2dd-293182453403 admin admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:24.907849 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd22511-edd9-468c-b2dd-293182453403 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:24.908414 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd22511-edd9-468c-b2dd-293182453403 admin admin] PATH_INFO: `/v3/projects/71fe5565b854460191eaa4fd12286bea/users/a8b0d4bdfaf345deadd2a4e28c8da2a1/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:24.913876 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd22511-edd9-468c-b2dd-293182453403 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71fe5565b854460191eaa4fd12286bea, user_id=a8b0d4bdfaf345deadd2a4e28c8da2a1, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:24.917703 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd22511-edd9-468c-b2dd-293182453403 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:24.933176 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 819/1634] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:13:24 2022] PUT /identity/v3/projects/71fe5565b854460191eaa4fd12286bea/users/a8b0d4bdfaf345deadd2a4e28c8da2a1/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:13:24.934547 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd22511-edd9-468c-b2dd-293182453403 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:24.945973 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52273fa8-78a2-44d2-b639-7d6d69fc1923 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:25.030502 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb08ea5-3e48-4c36-9ede-b59e42841919 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:25.031643 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb08ea5-3e48-4c36-9ede-b59e42841919 service cinder] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:25.032623 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb08ea5-3e48-4c36-9ede-b59e42841919 service cinder] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:25.033080 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb08ea5-3e48-4c36-9ede-b59e42841919 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:25.075833 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd22511-edd9-468c-b2dd-293182453403 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:25.084500 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3743ae6f-d969-4c92-be3f-5f2ba9b443c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:25.085928 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3743ae6f-d969-4c92-be3f-5f2ba9b443c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:25.086056 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3743ae6f-d969-4c92-be3f-5f2ba9b443c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:25.086669 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3743ae6f-d969-4c92-be3f-5f2ba9b443c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:25.088337 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3743ae6f-d969-4c92-be3f-5f2ba9b443c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:25.091270 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3743ae6f-d969-4c92-be3f-5f2ba9b443c3 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:25.120594 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 820/1635] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:13:24 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:13:25.120594 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cca486-1ec1-41af-b49f-922fb716974e None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:25.120966 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cca486-1ec1-41af-b49f-922fb716974e None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:25.121525 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cca486-1ec1-41af-b49f-922fb716974e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:25.144195 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb08ea5-3e48-4c36-9ede-b59e42841919 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:25.148620 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb08ea5-3e48-4c36-9ede-b59e42841919 service cinder] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:25.177739 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 816/1636] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:13:24 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:13:25.182030 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb08ea5-3e48-4c36-9ede-b59e42841919 service cinder] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:25.222245 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb08ea5-3e48-4c36-9ede-b59e42841919 service cinder] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:25.246651 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4f1c54-affe-4639-a1b1-c1854d122e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:25.249672 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f1c54-affe-4639-a1b1-c1854d122e7c admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:25.250458 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f1c54-affe-4639-a1b1-c1854d122e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:25.250796 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f1c54-affe-4639-a1b1-c1854d122e7c admin admin] PATH_INFO: `/v3/projects/978518a226d6471cba63f0cef4c857f6/users/2a35fbc2727346b5b0e19158119a753a/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:25.254332 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-91cca486-1ec1-41af-b49f-922fb716974e None None] MFA Rules not processed for user `a8b0d4bdfaf345deadd2a4e28c8da2a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:13:25.256808 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4f1c54-affe-4639-a1b1-c1854d122e7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=978518a226d6471cba63f0cef4c857f6, user_id=2a35fbc2727346b5b0e19158119a753a, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:25.256971 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-91cca486-1ec1-41af-b49f-922fb716974e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:25.266142 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4f1c54-affe-4639-a1b1-c1854d122e7c admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:25.275077 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:13:25.311171 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 817/1637] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:13:25 2022] PUT /identity/v3/projects/978518a226d6471cba63f0cef4c857f6/users/2a35fbc2727346b5b0e19158119a753a/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:13:25.313776 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12df38b-171d-44c2-963f-67d8d3972da1 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:25.314067 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12df38b-171d-44c2-963f-67d8d3972da1 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:25.317024 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12df38b-171d-44c2-963f-67d8d3972da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:25.366170 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:13:25.405379 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-f12df38b-171d-44c2-963f-67d8d3972da1 None None] MFA Rules not processed for user `2a35fbc2727346b5b0e19158119a753a`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:13:25.410956 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-f12df38b-171d-44c2-963f-67d8d3972da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:25.431153 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 821/1638] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:13:25 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:13:25.447370 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cca486-1ec1-41af-b49f-922fb716974e None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:25.479841 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cca486-1ec1-41af-b49f-922fb716974e None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:25.611622 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dffaea-17cc-4edd-9c91-282c7ef1e29d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:25.613761 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dffaea-17cc-4edd-9c91-282c7ef1e29d service cinder] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:25.614595 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dffaea-17cc-4edd-9c91-282c7ef1e29d service cinder] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:25.615942 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dffaea-17cc-4edd-9c91-282c7ef1e29d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:25.668767 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 818/1639] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:13:25 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:13:25.825393 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dffaea-17cc-4edd-9c91-282c7ef1e29d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:25.827144 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12df38b-171d-44c2-963f-67d8d3972da1 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:25.834901 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dffaea-17cc-4edd-9c91-282c7ef1e29d service cinder] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:25.847979 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12df38b-171d-44c2-963f-67d8d3972da1 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:25.864898 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2907192-018f-4b6d-8f21-659ebc28380b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:25.865641 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2907192-018f-4b6d-8f21-659ebc28380b service cinder] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:25.866653 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2907192-018f-4b6d-8f21-659ebc28380b service cinder] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:25.867866 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2907192-018f-4b6d-8f21-659ebc28380b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:25.903201 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 822/1640] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:13:25 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:13:25.925551 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2907192-018f-4b6d-8f21-659ebc28380b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:25.932300 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2907192-018f-4b6d-8f21-659ebc28380b service cinder] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:25.979791 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 819/1641] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:13:25 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 16:13:27.049535 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dffaea-17cc-4edd-9c91-282c7ef1e29d service cinder] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:27.075168 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dffaea-17cc-4edd-9c91-282c7ef1e29d service cinder] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:27.084522 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fc9017-3b2b-4d45-a720-bf4119fa3a52 tempest-VolumeQuotaClassesTest-757500807 tempest-VolumeQuotaClassesTest-757500807-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a35fbc2727346b5b0e19158119a753a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '978518a226d6471cba63f0cef4c857f6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:27.085594 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fc9017-3b2b-4d45-a720-bf4119fa3a52 tempest-VolumeQuotaClassesTest-757500807 tempest-VolumeQuotaClassesTest-757500807-project-admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:27.086101 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fc9017-3b2b-4d45-a720-bf4119fa3a52 tempest-VolumeQuotaClassesTest-757500807 tempest-VolumeQuotaClassesTest-757500807-project-admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:27.086698 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fc9017-3b2b-4d45-a720-bf4119fa3a52 tempest-VolumeQuotaClassesTest-757500807 tempest-VolumeQuotaClassesTest-757500807-project-admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:27.087963 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fc9017-3b2b-4d45-a720-bf4119fa3a52 tempest-VolumeQuotaClassesTest-757500807 tempest-VolumeQuotaClassesTest-757500807-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:13:27.089616 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fc9017-3b2b-4d45-a720-bf4119fa3a52 tempest-VolumeQuotaClassesTest-757500807 tempest-VolumeQuotaClassesTest-757500807-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:27.094820 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fc9017-3b2b-4d45-a720-bf4119fa3a52 tempest-VolumeQuotaClassesTest-757500807 tempest-VolumeQuotaClassesTest-757500807-project-admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:27.177480 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 823/1642] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:13:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:13:27.184549 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2907192-018f-4b6d-8f21-659ebc28380b service cinder] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:27.200689 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2907192-018f-4b6d-8f21-659ebc28380b service cinder] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:27.222589 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e3e162-2b14-46d4-be0e-258714f4e5e8 tempest-VolumeQuotaClassesTest-757500807 tempest-VolumeQuotaClassesTest-757500807-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a35fbc2727346b5b0e19158119a753a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '978518a226d6471cba63f0cef4c857f6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:27.223812 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e3e162-2b14-46d4-be0e-258714f4e5e8 tempest-VolumeQuotaClassesTest-757500807 tempest-VolumeQuotaClassesTest-757500807-project-admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:27.225397 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e3e162-2b14-46d4-be0e-258714f4e5e8 tempest-VolumeQuotaClassesTest-757500807 tempest-VolumeQuotaClassesTest-757500807-project-admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:27.225670 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e3e162-2b14-46d4-be0e-258714f4e5e8 tempest-VolumeQuotaClassesTest-757500807 tempest-VolumeQuotaClassesTest-757500807-project-admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:27.227337 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e3e162-2b14-46d4-be0e-258714f4e5e8 tempest-VolumeQuotaClassesTest-757500807 tempest-VolumeQuotaClassesTest-757500807-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:27.238118 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e3e162-2b14-46d4-be0e-258714f4e5e8 tempest-VolumeQuotaClassesTest-757500807 tempest-VolumeQuotaClassesTest-757500807-project-admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:27.358941 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 820/1643] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:13:27 2022] POST /identity/v3/projects => generated 355 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:13:27.363243 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fc9017-3b2b-4d45-a720-bf4119fa3a52 tempest-VolumeQuotaClassesTest-757500807 tempest-VolumeQuotaClassesTest-757500807-project-admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:27.380784 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fc9017-3b2b-4d45-a720-bf4119fa3a52 tempest-VolumeQuotaClassesTest-757500807 tempest-VolumeQuotaClassesTest-757500807-project-admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:27.390884 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e3e162-2b14-46d4-be0e-258714f4e5e8 tempest-VolumeQuotaClassesTest-757500807 tempest-VolumeQuotaClassesTest-757500807-project-admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:27.560173 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d75638-e2aa-4252-bb04-4f3843994d56 tempest-VolumeQuotaClassesTest-757500807 tempest-VolumeQuotaClassesTest-757500807-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a35fbc2727346b5b0e19158119a753a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '978518a226d6471cba63f0cef4c857f6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:27.560821 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d75638-e2aa-4252-bb04-4f3843994d56 tempest-VolumeQuotaClassesTest-757500807 tempest-VolumeQuotaClassesTest-757500807-project-admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:27.561615 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d75638-e2aa-4252-bb04-4f3843994d56 tempest-VolumeQuotaClassesTest-757500807 tempest-VolumeQuotaClassesTest-757500807-project-admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:27.561907 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d75638-e2aa-4252-bb04-4f3843994d56 tempest-VolumeQuotaClassesTest-757500807 tempest-VolumeQuotaClassesTest-757500807-project-admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:27.562626 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d75638-e2aa-4252-bb04-4f3843994d56 tempest-VolumeQuotaClassesTest-757500807 tempest-VolumeQuotaClassesTest-757500807-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:13:27.562935 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d75638-e2aa-4252-bb04-4f3843994d56 tempest-VolumeQuotaClassesTest-757500807 tempest-VolumeQuotaClassesTest-757500807-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:27.564582 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d75638-e2aa-4252-bb04-4f3843994d56 tempest-VolumeQuotaClassesTest-757500807 tempest-VolumeQuotaClassesTest-757500807-project-admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:27.580445 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e3e162-2b14-46d4-be0e-258714f4e5e8 tempest-VolumeQuotaClassesTest-757500807 tempest-VolumeQuotaClassesTest-757500807-project-admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:27.596079 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4079cb98-2367-49b7-a11e-9a891bd3f13d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:27.596079 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4079cb98-2367-49b7-a11e-9a891bd3f13d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:27.596458 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4079cb98-2367-49b7-a11e-9a891bd3f13d admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:27.596801 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4079cb98-2367-49b7-a11e-9a891bd3f13d admin admin] PATH_INFO: `/v3/users/597611fe2b3e44d5b9f24ae1158e49d3` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:27.599150 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4079cb98-2367-49b7-a11e-9a891bd3f13d admin admin] RBAC: Authorizing `identity:delete_user(user_id=597611fe2b3e44d5b9f24ae1158e49d3)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:27.600468 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: WARNING py.warnings [None req-4079cb98-2367-49b7-a11e-9a891bd3f13d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 16:13:27.600468 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: warnings.warn(msg) Sep 19 16:13:27.600468 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: Sep 19 16:13:27.605427 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4079cb98-2367-49b7-a11e-9a891bd3f13d admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:27.614445 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 824/1644] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:13:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:13:27.726412 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d75638-e2aa-4252-bb04-4f3843994d56 tempest-VolumeQuotaClassesTest-757500807 tempest-VolumeQuotaClassesTest-757500807-project-admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:27.740953 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:13:27.756604 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d75638-e2aa-4252-bb04-4f3843994d56 tempest-VolumeQuotaClassesTest-757500807 tempest-VolumeQuotaClassesTest-757500807-project-admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:27.805580 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-4079cb98-2367-49b7-a11e-9a891bd3f13d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '597611fe2b3e44d5b9f24ae1158e49d3'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:27.808786 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:13:27.810642 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002549e4-3393-468c-a7a2-b3ade0f094c2 tempest-VolumeQuotaClassesTest-757500807 tempest-VolumeQuotaClassesTest-757500807-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a35fbc2727346b5b0e19158119a753a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '978518a226d6471cba63f0cef4c857f6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:27.817120 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002549e4-3393-468c-a7a2-b3ade0f094c2 tempest-VolumeQuotaClassesTest-757500807 tempest-VolumeQuotaClassesTest-757500807-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:27.818065 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002549e4-3393-468c-a7a2-b3ade0f094c2 tempest-VolumeQuotaClassesTest-757500807 tempest-VolumeQuotaClassesTest-757500807-project-admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:27.819077 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002549e4-3393-468c-a7a2-b3ade0f094c2 tempest-VolumeQuotaClassesTest-757500807 tempest-VolumeQuotaClassesTest-757500807-project-admin] PATH_INFO: `/v3/projects/fa65231f00364c09970d00b34fa1a165` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:27.822516 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002549e4-3393-468c-a7a2-b3ade0f094c2 tempest-VolumeQuotaClassesTest-757500807 tempest-VolumeQuotaClassesTest-757500807-project-admin] RBAC: Authorizing `identity:delete_project(project_id=fa65231f00364c09970d00b34fa1a165)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:27.826481 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: WARNING py.warnings [None req-002549e4-3393-468c-a7a2-b3ade0f094c2 tempest-VolumeQuotaClassesTest-757500807 tempest-VolumeQuotaClassesTest-757500807-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 16:13:27.826481 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: warnings.warn(msg) Sep 19 16:13:27.826481 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: Sep 19 16:13:27.829866 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-4079cb98-2367-49b7-a11e-9a891bd3f13d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '597611fe2b3e44d5b9f24ae1158e49d3'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:27.830215 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002549e4-3393-468c-a7a2-b3ade0f094c2 tempest-VolumeQuotaClassesTest-757500807 tempest-VolumeQuotaClassesTest-757500807-project-admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:27.872525 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-4079cb98-2367-49b7-a11e-9a891bd3f13d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '597611fe2b3e44d5b9f24ae1158e49d3'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:27.890681 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-4079cb98-2367-49b7-a11e-9a891bd3f13d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '597611fe2b3e44d5b9f24ae1158e49d3'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:27.891341 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-4079cb98-2367-49b7-a11e-9a891bd3f13d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '597611fe2b3e44d5b9f24ae1158e49d3'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:27.924968 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 821/1645] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:13:27 2022] DELETE /identity/v3/users/597611fe2b3e44d5b9f24ae1158e49d3 => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 16:13:27.926933 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a13a54-599a-46d0-ab5b-7881cfccd54b None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:27.927267 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a13a54-599a-46d0-ab5b-7881cfccd54b None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:27.927561 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a13a54-599a-46d0-ab5b-7881cfccd54b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:27.986277 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-e0a13a54-599a-46d0-ab5b-7881cfccd54b None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:13:27.991058 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-e0a13a54-599a-46d0-ab5b-7881cfccd54b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:28.001956 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-002549e4-3393-468c-a7a2-b3ade0f094c2 tempest-VolumeQuotaClassesTest-757500807 tempest-VolumeQuotaClassesTest-757500807-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa65231f00364c09970d00b34fa1a165', 'request_id': 'req-002549e4-3393-468c-a7a2-b3ade0f094c2'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:28.070922 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-002549e4-3393-468c-a7a2-b3ade0f094c2 tempest-VolumeQuotaClassesTest-757500807 tempest-VolumeQuotaClassesTest-757500807-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa65231f00364c09970d00b34fa1a165', 'request_id': 'req-002549e4-3393-468c-a7a2-b3ade0f094c2'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:28.111410 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-002549e4-3393-468c-a7a2-b3ade0f094c2 tempest-VolumeQuotaClassesTest-757500807 tempest-VolumeQuotaClassesTest-757500807-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa65231f00364c09970d00b34fa1a165', 'request_id': 'req-002549e4-3393-468c-a7a2-b3ade0f094c2'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:28.170576 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-002549e4-3393-468c-a7a2-b3ade0f094c2 tempest-VolumeQuotaClassesTest-757500807 tempest-VolumeQuotaClassesTest-757500807-project-admin] The token cache is being invalidate because project fa65231f00364c09970d00b34fa1a165 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81617) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:13:28.170879 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-002549e4-3393-468c-a7a2-b3ade0f094c2 tempest-VolumeQuotaClassesTest-757500807 tempest-VolumeQuotaClassesTest-757500807-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:28.172426 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-002549e4-3393-468c-a7a2-b3ade0f094c2 tempest-VolumeQuotaClassesTest-757500807 tempest-VolumeQuotaClassesTest-757500807-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:28.174440 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 825/1646] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:13:27 2022] DELETE /identity/v3/projects/fa65231f00364c09970d00b34fa1a165 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 16:13:28.285991 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 822/1647] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:13:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:13:28.333154 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002549e4-3393-468c-a7a2-b3ade0f094c2 tempest-VolumeQuotaClassesTest-757500807 tempest-VolumeQuotaClassesTest-757500807-project-admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:28.334797 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-002549e4-3393-468c-a7a2-b3ade0f094c2 tempest-VolumeQuotaClassesTest-757500807 tempest-VolumeQuotaClassesTest-757500807-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:28.606000 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002549e4-3393-468c-a7a2-b3ade0f094c2 tempest-VolumeQuotaClassesTest-757500807 tempest-VolumeQuotaClassesTest-757500807-project-admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:28.628186 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e161279-3679-4df7-95b5-38e170dc3e74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:28.631136 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e161279-3679-4df7-95b5-38e170dc3e74 service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:28.631877 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e161279-3679-4df7-95b5-38e170dc3e74 service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:28.636233 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e161279-3679-4df7-95b5-38e170dc3e74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:28.643959 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a13a54-599a-46d0-ab5b-7881cfccd54b None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:28.645592 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-e0a13a54-599a-46d0-ab5b-7881cfccd54b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:28.673328 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e161279-3679-4df7-95b5-38e170dc3e74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:28.674841 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e161279-3679-4df7-95b5-38e170dc3e74 service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:28.694077 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a13a54-599a-46d0-ab5b-7881cfccd54b None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:28.699068 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 826/1648] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:13:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:13:28.711128 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98928734-c23d-47fb-8dec-71893c7b8f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:28.713974 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98928734-c23d-47fb-8dec-71893c7b8f53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:28.715071 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98928734-c23d-47fb-8dec-71893c7b8f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:28.716237 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98928734-c23d-47fb-8dec-71893c7b8f53 admin admin] PATH_INFO: `/v3/users/6e24f66ae3704054a89cc44eaf9663ff` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:28.725634 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98928734-c23d-47fb-8dec-71893c7b8f53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e24f66ae3704054a89cc44eaf9663ff)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:28.732836 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98928734-c23d-47fb-8dec-71893c7b8f53 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:28.912779 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-98928734-c23d-47fb-8dec-71893c7b8f53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e24f66ae3704054a89cc44eaf9663ff'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:28.942805 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-98928734-c23d-47fb-8dec-71893c7b8f53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e24f66ae3704054a89cc44eaf9663ff'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:28.993155 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-98928734-c23d-47fb-8dec-71893c7b8f53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e24f66ae3704054a89cc44eaf9663ff'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:29.009484 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-98928734-c23d-47fb-8dec-71893c7b8f53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e24f66ae3704054a89cc44eaf9663ff'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:29.010354 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-98928734-c23d-47fb-8dec-71893c7b8f53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e24f66ae3704054a89cc44eaf9663ff'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:29.025180 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 823/1649] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:13:28 2022] DELETE /identity/v3/users/6e24f66ae3704054a89cc44eaf9663ff => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:13:29.032324 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384bfc21-c42d-4fb5-85f3-0fca8ea82842 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:29.033232 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384bfc21-c42d-4fb5-85f3-0fca8ea82842 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:29.034002 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384bfc21-c42d-4fb5-85f3-0fca8ea82842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:29.188838 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-384bfc21-c42d-4fb5-85f3-0fca8ea82842 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:13:29.198339 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-384bfc21-c42d-4fb5-85f3-0fca8ea82842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:29.585135 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 827/1650] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:13:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 555 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:13:29.594107 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98928734-c23d-47fb-8dec-71893c7b8f53 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:29.595648 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-98928734-c23d-47fb-8dec-71893c7b8f53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:29.607629 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384bfc21-c42d-4fb5-85f3-0fca8ea82842 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:29.610157 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-384bfc21-c42d-4fb5-85f3-0fca8ea82842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:29.621209 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98928734-c23d-47fb-8dec-71893c7b8f53 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:29.645120 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc511d95-04d2-4d7e-87f5-b0053861d911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:29.645805 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc511d95-04d2-4d7e-87f5-b0053861d911 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:29.646087 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc511d95-04d2-4d7e-87f5-b0053861d911 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:29.646359 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc511d95-04d2-4d7e-87f5-b0053861d911 admin admin] PATH_INFO: `/v3/users/95f43fa305e84ebdb268de29df725737` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:29.653457 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc511d95-04d2-4d7e-87f5-b0053861d911 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95f43fa305e84ebdb268de29df725737)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:29.656238 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc511d95-04d2-4d7e-87f5-b0053861d911 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:29.720696 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:13:29.767243 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:13:29.793572 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384bfc21-c42d-4fb5-85f3-0fca8ea82842 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:29.800137 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-dc511d95-04d2-4d7e-87f5-b0053861d911 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95f43fa305e84ebdb268de29df725737'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:29.807519 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e421d07d-8d15-4f11-a73f-87edf7932590 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:29.810523 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e421d07d-8d15-4f11-a73f-87edf7932590 service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:29.810768 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-dc511d95-04d2-4d7e-87f5-b0053861d911 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95f43fa305e84ebdb268de29df725737'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:29.811857 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e421d07d-8d15-4f11-a73f-87edf7932590 service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:29.812777 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e421d07d-8d15-4f11-a73f-87edf7932590 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:29.827348 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-dc511d95-04d2-4d7e-87f5-b0053861d911 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95f43fa305e84ebdb268de29df725737'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:29.841419 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-dc511d95-04d2-4d7e-87f5-b0053861d911 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95f43fa305e84ebdb268de29df725737'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:29.842215 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-dc511d95-04d2-4d7e-87f5-b0053861d911 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95f43fa305e84ebdb268de29df725737'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:29.872416 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 824/1651] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:13:29 2022] DELETE /identity/v3/users/95f43fa305e84ebdb268de29df725737 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 16:13:29.875521 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a774450-9236-4cde-b669-3fab3db97256 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:29.875800 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a774450-9236-4cde-b669-3fab3db97256 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:29.876077 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a774450-9236-4cde-b669-3fab3db97256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:29.886764 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e421d07d-8d15-4f11-a73f-87edf7932590 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:29.893893 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e421d07d-8d15-4f11-a73f-87edf7932590 service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:29.928570 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 828/1652] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:13:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 16:13:29.934461 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-0a774450-9236-4cde-b669-3fab3db97256 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:13:29.935906 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-0a774450-9236-4cde-b669-3fab3db97256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:30.063383 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 825/1653] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:13:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:13:30.132832 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e421d07d-8d15-4f11-a73f-87edf7932590 service neutron] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:30.156617 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e421d07d-8d15-4f11-a73f-87edf7932590 service neutron] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:30.300630 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e505a87-88db-45a0-893c-60ee13b59085 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:30.303954 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e505a87-88db-45a0-893c-60ee13b59085 service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:30.304649 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e505a87-88db-45a0-893c-60ee13b59085 service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:30.305159 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e505a87-88db-45a0-893c-60ee13b59085 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:30.323386 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e505a87-88db-45a0-893c-60ee13b59085 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:30.325636 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e505a87-88db-45a0-893c-60ee13b59085 service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:30.355885 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 829/1654] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:13:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:13:31.433215 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a774450-9236-4cde-b669-3fab3db97256 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:31.434958 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-0a774450-9236-4cde-b669-3fab3db97256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:31.457094 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a774450-9236-4cde-b669-3fab3db97256 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:31.471540 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7caefba-4146-4ada-aaf2-528389ec60c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:31.472210 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7caefba-4146-4ada-aaf2-528389ec60c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:31.472493 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7caefba-4146-4ada-aaf2-528389ec60c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:31.472767 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7caefba-4146-4ada-aaf2-528389ec60c5 admin admin] PATH_INFO: `/v3/projects/2dc0c57ce73a415990562e3113a7b316` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:31.474068 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7caefba-4146-4ada-aaf2-528389ec60c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2dc0c57ce73a415990562e3113a7b316)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:31.477041 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7caefba-4146-4ada-aaf2-528389ec60c5 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:31.773404 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-b7caefba-4146-4ada-aaf2-528389ec60c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2dc0c57ce73a415990562e3113a7b316', 'request_id': 'req-b7caefba-4146-4ada-aaf2-528389ec60c5'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:31.786054 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-b7caefba-4146-4ada-aaf2-528389ec60c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2dc0c57ce73a415990562e3113a7b316', 'request_id': 'req-b7caefba-4146-4ada-aaf2-528389ec60c5'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:31.795873 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-b7caefba-4146-4ada-aaf2-528389ec60c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2dc0c57ce73a415990562e3113a7b316', 'request_id': 'req-b7caefba-4146-4ada-aaf2-528389ec60c5'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:31.827671 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-b7caefba-4146-4ada-aaf2-528389ec60c5 admin admin] The token cache is being invalidate because project 2dc0c57ce73a415990562e3113a7b316 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81618) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:13:31.828200 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-b7caefba-4146-4ada-aaf2-528389ec60c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:31.829680 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-b7caefba-4146-4ada-aaf2-528389ec60c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:31.839393 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e505a87-88db-45a0-893c-60ee13b59085 service neutron] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:31.844929 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 826/1655] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:13:31 2022] DELETE /identity/v3/projects/2dc0c57ce73a415990562e3113a7b316 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:13:31.855250 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e505a87-88db-45a0-893c-60ee13b59085 service neutron] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:31.994804 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87941e7a-a05b-452e-bc63-7d18ea93bfd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:31.996715 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87941e7a-a05b-452e-bc63-7d18ea93bfd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:31.996990 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87941e7a-a05b-452e-bc63-7d18ea93bfd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:31.999739 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87941e7a-a05b-452e-bc63-7d18ea93bfd2 admin admin] PATH_INFO: `/v3/users/a8b0d4bdfaf345deadd2a4e28c8da2a1` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:32.004257 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87941e7a-a05b-452e-bc63-7d18ea93bfd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8b0d4bdfaf345deadd2a4e28c8da2a1)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:32.007319 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87941e7a-a05b-452e-bc63-7d18ea93bfd2 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:32.170726 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-87941e7a-a05b-452e-bc63-7d18ea93bfd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8b0d4bdfaf345deadd2a4e28c8da2a1'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:32.176132 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-87941e7a-a05b-452e-bc63-7d18ea93bfd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8b0d4bdfaf345deadd2a4e28c8da2a1'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:32.213431 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-87941e7a-a05b-452e-bc63-7d18ea93bfd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8b0d4bdfaf345deadd2a4e28c8da2a1'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:32.231095 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7caefba-4146-4ada-aaf2-528389ec60c5 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:32.233537 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-b7caefba-4146-4ada-aaf2-528389ec60c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:32.262447 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-87941e7a-a05b-452e-bc63-7d18ea93bfd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8b0d4bdfaf345deadd2a4e28c8da2a1'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:32.267539 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7caefba-4146-4ada-aaf2-528389ec60c5 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:32.268558 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-87941e7a-a05b-452e-bc63-7d18ea93bfd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8b0d4bdfaf345deadd2a4e28c8da2a1'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:32.284933 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:13:32.296579 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c83d73-5659-4d85-a7b0-fcca9cd2179b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:32.297641 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c83d73-5659-4d85-a7b0-fcca9cd2179b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:32.297817 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c83d73-5659-4d85-a7b0-fcca9cd2179b admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:32.298145 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c83d73-5659-4d85-a7b0-fcca9cd2179b admin admin] PATH_INFO: `/v3/projects/0350ad6e81224f0b801874c75828e039` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:32.300249 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c83d73-5659-4d85-a7b0-fcca9cd2179b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0350ad6e81224f0b801874c75828e039)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:32.314247 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 830/1656] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:13:31 2022] DELETE /identity/v3/users/a8b0d4bdfaf345deadd2a4e28c8da2a1 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:13:32.316187 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c83d73-5659-4d85-a7b0-fcca9cd2179b admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:32.400139 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:13:32.427987 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-f3c83d73-5659-4d85-a7b0-fcca9cd2179b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0350ad6e81224f0b801874c75828e039', 'request_id': 'req-f3c83d73-5659-4d85-a7b0-fcca9cd2179b'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:32.438968 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-f3c83d73-5659-4d85-a7b0-fcca9cd2179b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0350ad6e81224f0b801874c75828e039', 'request_id': 'req-f3c83d73-5659-4d85-a7b0-fcca9cd2179b'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:32.454840 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-f3c83d73-5659-4d85-a7b0-fcca9cd2179b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0350ad6e81224f0b801874c75828e039', 'request_id': 'req-f3c83d73-5659-4d85-a7b0-fcca9cd2179b'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:32.473024 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-f3c83d73-5659-4d85-a7b0-fcca9cd2179b admin admin] The token cache is being invalidate because project 0350ad6e81224f0b801874c75828e039 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81618) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:13:32.473755 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-f3c83d73-5659-4d85-a7b0-fcca9cd2179b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:32.475695 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-f3c83d73-5659-4d85-a7b0-fcca9cd2179b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:32.478470 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 827/1657] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:13:32 2022] DELETE /identity/v3/projects/0350ad6e81224f0b801874c75828e039 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:13:32.484317 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87941e7a-a05b-452e-bc63-7d18ea93bfd2 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:32.490955 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-87941e7a-a05b-452e-bc63-7d18ea93bfd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:32.666603 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87941e7a-a05b-452e-bc63-7d18ea93bfd2 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:32.832193 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48a098e-a5e3-41ad-ad62-0f1853bcbbd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:32.833215 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a098e-a5e3-41ad-ad62-0f1853bcbbd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:32.833606 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a098e-a5e3-41ad-ad62-0f1853bcbbd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:32.834076 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a098e-a5e3-41ad-ad62-0f1853bcbbd0 admin admin] PATH_INFO: `/v3/users/2a35fbc2727346b5b0e19158119a753a` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:32.840419 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48a098e-a5e3-41ad-ad62-0f1853bcbbd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a35fbc2727346b5b0e19158119a753a)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:32.843913 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48a098e-a5e3-41ad-ad62-0f1853bcbbd0 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:33.022937 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c83d73-5659-4d85-a7b0-fcca9cd2179b admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:33.025082 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-f3c83d73-5659-4d85-a7b0-fcca9cd2179b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:33.054668 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-a48a098e-a5e3-41ad-ad62-0f1853bcbbd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a35fbc2727346b5b0e19158119a753a'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:33.055283 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c83d73-5659-4d85-a7b0-fcca9cd2179b admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:33.057011 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-a48a098e-a5e3-41ad-ad62-0f1853bcbbd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a35fbc2727346b5b0e19158119a753a'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:33.076638 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdda67d-a973-4c4f-8fcd-ae45ff454c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:33.077316 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdda67d-a973-4c4f-8fcd-ae45ff454c8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:33.077603 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdda67d-a973-4c4f-8fcd-ae45ff454c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:33.078086 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdda67d-a973-4c4f-8fcd-ae45ff454c8f admin admin] PATH_INFO: `/v3/projects/8c22de2ade474ef3a075af2ea29cc730` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:33.086060 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdda67d-a973-4c4f-8fcd-ae45ff454c8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c22de2ade474ef3a075af2ea29cc730)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:33.088727 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdda67d-a973-4c4f-8fcd-ae45ff454c8f admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:33.102092 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-a48a098e-a5e3-41ad-ad62-0f1853bcbbd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a35fbc2727346b5b0e19158119a753a'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:33.214817 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-a48a098e-a5e3-41ad-ad62-0f1853bcbbd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a35fbc2727346b5b0e19158119a753a'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:33.219124 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-a48a098e-a5e3-41ad-ad62-0f1853bcbbd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a35fbc2727346b5b0e19158119a753a'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:33.250833 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-4cdda67d-a973-4c4f-8fcd-ae45ff454c8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c22de2ade474ef3a075af2ea29cc730', 'request_id': 'req-4cdda67d-a973-4c4f-8fcd-ae45ff454c8f'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:33.281982 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 831/1658] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:13:32 2022] DELETE /identity/v3/users/2a35fbc2727346b5b0e19158119a753a => generated 0 bytes in 798 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:13:33.294920 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-4cdda67d-a973-4c4f-8fcd-ae45ff454c8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c22de2ade474ef3a075af2ea29cc730', 'request_id': 'req-4cdda67d-a973-4c4f-8fcd-ae45ff454c8f'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:33.331649 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-4cdda67d-a973-4c4f-8fcd-ae45ff454c8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c22de2ade474ef3a075af2ea29cc730', 'request_id': 'req-4cdda67d-a973-4c4f-8fcd-ae45ff454c8f'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:33.380726 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-4cdda67d-a973-4c4f-8fcd-ae45ff454c8f admin admin] The token cache is being invalidate because project 8c22de2ade474ef3a075af2ea29cc730 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81618) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:13:33.381281 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-4cdda67d-a973-4c4f-8fcd-ae45ff454c8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:33.400751 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-4cdda67d-a973-4c4f-8fcd-ae45ff454c8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:33.404935 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 828/1659] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:13:33 2022] DELETE /identity/v3/projects/8c22de2ade474ef3a075af2ea29cc730 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:13:33.410947 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48a098e-a5e3-41ad-ad62-0f1853bcbbd0 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:33.412587 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-a48a098e-a5e3-41ad-ad62-0f1853bcbbd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:33.472822 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:13:33.575830 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48a098e-a5e3-41ad-ad62-0f1853bcbbd0 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:33.589760 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b74f10-de24-4093-a893-cc418890223c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:33.590575 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b74f10-de24-4093-a893-cc418890223c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:33.590937 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b74f10-de24-4093-a893-cc418890223c admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:33.591290 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b74f10-de24-4093-a893-cc418890223c admin admin] PATH_INFO: `/v3/users/a321283a551b47de86255f93830017e6` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:33.595510 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b74f10-de24-4093-a893-cc418890223c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a321283a551b47de86255f93830017e6)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:33.597707 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b74f10-de24-4093-a893-cc418890223c admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:33.744135 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-f1b74f10-de24-4093-a893-cc418890223c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a321283a551b47de86255f93830017e6'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:33.744855 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-f1b74f10-de24-4093-a893-cc418890223c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a321283a551b47de86255f93830017e6'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:33.765509 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-f1b74f10-de24-4093-a893-cc418890223c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a321283a551b47de86255f93830017e6'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:33.834677 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-f1b74f10-de24-4093-a893-cc418890223c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a321283a551b47de86255f93830017e6'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:33.835511 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-f1b74f10-de24-4093-a893-cc418890223c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a321283a551b47de86255f93830017e6'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:33.889101 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 832/1660] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:13:33 2022] DELETE /identity/v3/users/a321283a551b47de86255f93830017e6 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:13:36.905726 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c4079d-4964-41ad-a454-6d3b86f1ec0f None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:36.906214 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c4079d-4964-41ad-a454-6d3b86f1ec0f None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:36.906740 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c4079d-4964-41ad-a454-6d3b86f1ec0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:36.987096 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-b6c4079d-4964-41ad-a454-6d3b86f1ec0f None None] MFA Rules not processed for user `fd36089b3c2141218660ab2458376798`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:13:36.988396 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-b6c4079d-4964-41ad-a454-6d3b86f1ec0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:37.087262 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b74f10-de24-4093-a893-cc418890223c admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:37.088762 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-f1b74f10-de24-4093-a893-cc418890223c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:37.096474 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 829/1661] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:13:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:13:37.130421 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89a359c-ae13-4fba-b4cc-9da1b767babf None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:37.130730 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89a359c-ae13-4fba-b4cc-9da1b767babf None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:37.131014 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89a359c-ae13-4fba-b4cc-9da1b767babf None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:37.132750 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 830/1662] 10.222.0.57 () {54 vars in 965 bytes} [Mon Sep 19 16:13:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:13:37.185514 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1675b2f-ffdb-475c-9188-9579a50dd899 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:37.185802 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1675b2f-ffdb-475c-9188-9579a50dd899 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:37.186079 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1675b2f-ffdb-475c-9188-9579a50dd899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:37.282465 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b74f10-de24-4093-a893-cc418890223c admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:37.298685 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-d1675b2f-ffdb-475c-9188-9579a50dd899 None None] MFA Rules not processed for user `1ecd01cc72834b72b5ff2c4b47249737`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:13:37.299140 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1357159a-636c-4924-8663-9d5ce49ee7bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:37.299967 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1357159a-636c-4924-8663-9d5ce49ee7bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:37.300390 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1357159a-636c-4924-8663-9d5ce49ee7bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:37.300806 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1357159a-636c-4924-8663-9d5ce49ee7bd admin admin] PATH_INFO: `/v3/projects/978518a226d6471cba63f0cef4c857f6` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:37.302618 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-d1675b2f-ffdb-475c-9188-9579a50dd899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:37.307518 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1357159a-636c-4924-8663-9d5ce49ee7bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=978518a226d6471cba63f0cef4c857f6)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:37.312538 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1357159a-636c-4924-8663-9d5ce49ee7bd admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:37.332803 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:13:37.468646 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:13:37.479023 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-1357159a-636c-4924-8663-9d5ce49ee7bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '978518a226d6471cba63f0cef4c857f6', 'request_id': 'req-1357159a-636c-4924-8663-9d5ce49ee7bd'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:37.491209 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-1357159a-636c-4924-8663-9d5ce49ee7bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '978518a226d6471cba63f0cef4c857f6', 'request_id': 'req-1357159a-636c-4924-8663-9d5ce49ee7bd'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:37.509923 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-1357159a-636c-4924-8663-9d5ce49ee7bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '978518a226d6471cba63f0cef4c857f6', 'request_id': 'req-1357159a-636c-4924-8663-9d5ce49ee7bd'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:37.521803 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 831/1663] 10.222.0.57 () {58 vars in 1062 bytes} [Mon Sep 19 16:13:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:13:37.523785 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1675b2f-ffdb-475c-9188-9579a50dd899 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:37.529084 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-d1675b2f-ffdb-475c-9188-9579a50dd899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:37.542616 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-1357159a-636c-4924-8663-9d5ce49ee7bd admin admin] The token cache is being invalidate because project 978518a226d6471cba63f0cef4c857f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81617) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:13:37.542928 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-1357159a-636c-4924-8663-9d5ce49ee7bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:37.543624 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-1357159a-636c-4924-8663-9d5ce49ee7bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:37.551573 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 833/1664] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:13:37 2022] DELETE /identity/v3/projects/978518a226d6471cba63f0cef4c857f6 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:13:37.552592 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1357159a-636c-4924-8663-9d5ce49ee7bd admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:37.554102 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-1357159a-636c-4924-8663-9d5ce49ee7bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:37.566725 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1675b2f-ffdb-475c-9188-9579a50dd899 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:37.731804 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1357159a-636c-4924-8663-9d5ce49ee7bd admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:37.753578 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176f635d-daff-4471-bded-173cd5d13bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:37.755713 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176f635d-daff-4471-bded-173cd5d13bf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:37.757383 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176f635d-daff-4471-bded-173cd5d13bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:37.761212 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176f635d-daff-4471-bded-173cd5d13bf2 admin admin] PATH_INFO: `/v3/projects/71fe5565b854460191eaa4fd12286bea` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:37.777306 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176f635d-daff-4471-bded-173cd5d13bf2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71fe5565b854460191eaa4fd12286bea)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:37.789699 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176f635d-daff-4471-bded-173cd5d13bf2 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:37.823096 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c999e-8810-4e1b-acd6-8aff7294f2a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:37.825426 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c999e-8810-4e1b-acd6-8aff7294f2a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:37.828589 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c999e-8810-4e1b-acd6-8aff7294f2a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:37.829473 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c999e-8810-4e1b-acd6-8aff7294f2a5 admin admin] PATH_INFO: `/v3/projects/e1468393e72147489fcf7e30df416868` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:37.837062 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3c999e-8810-4e1b-acd6-8aff7294f2a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1468393e72147489fcf7e30df416868)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:37.854334 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3c999e-8810-4e1b-acd6-8aff7294f2a5 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:37.924162 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-176f635d-daff-4471-bded-173cd5d13bf2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71fe5565b854460191eaa4fd12286bea', 'request_id': 'req-176f635d-daff-4471-bded-173cd5d13bf2'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:37.940034 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-176f635d-daff-4471-bded-173cd5d13bf2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71fe5565b854460191eaa4fd12286bea', 'request_id': 'req-176f635d-daff-4471-bded-173cd5d13bf2'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:37.955849 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-176f635d-daff-4471-bded-173cd5d13bf2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71fe5565b854460191eaa4fd12286bea', 'request_id': 'req-176f635d-daff-4471-bded-173cd5d13bf2'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:37.982442 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-176f635d-daff-4471-bded-173cd5d13bf2 admin admin] The token cache is being invalidate because project 71fe5565b854460191eaa4fd12286bea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81617) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:13:37.982902 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-176f635d-daff-4471-bded-173cd5d13bf2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:37.984204 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-176f635d-daff-4471-bded-173cd5d13bf2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:37.989011 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 834/1665] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:13:37 2022] DELETE /identity/v3/projects/71fe5565b854460191eaa4fd12286bea => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:13:37.992068 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176f635d-daff-4471-bded-173cd5d13bf2 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:37.997873 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-176f635d-daff-4471-bded-173cd5d13bf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:38.059650 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-ce3c999e-8810-4e1b-acd6-8aff7294f2a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1468393e72147489fcf7e30df416868', 'request_id': 'req-ce3c999e-8810-4e1b-acd6-8aff7294f2a5'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:38.076331 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-ce3c999e-8810-4e1b-acd6-8aff7294f2a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1468393e72147489fcf7e30df416868', 'request_id': 'req-ce3c999e-8810-4e1b-acd6-8aff7294f2a5'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:38.091594 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-ce3c999e-8810-4e1b-acd6-8aff7294f2a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1468393e72147489fcf7e30df416868', 'request_id': 'req-ce3c999e-8810-4e1b-acd6-8aff7294f2a5'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:38.121446 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-ce3c999e-8810-4e1b-acd6-8aff7294f2a5 admin admin] The token cache is being invalidate because project e1468393e72147489fcf7e30df416868 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81618) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:13:38.121892 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-ce3c999e-8810-4e1b-acd6-8aff7294f2a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:38.123431 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-ce3c999e-8810-4e1b-acd6-8aff7294f2a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:38.126774 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 832/1666] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:13:37 2022] DELETE /identity/v3/projects/e1468393e72147489fcf7e30df416868 => generated 0 bytes in 604 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:13:38.129747 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68642835-b66b-44f1-9a5e-00480064dd8b None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:38.130292 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68642835-b66b-44f1-9a5e-00480064dd8b None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:38.130854 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68642835-b66b-44f1-9a5e-00480064dd8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:38.201148 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-68642835-b66b-44f1-9a5e-00480064dd8b None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:13:38.202190 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:13:38.205308 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-68642835-b66b-44f1-9a5e-00480064dd8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:38.248684 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176f635d-daff-4471-bded-173cd5d13bf2 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:38.303981 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:13:38.392952 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 833/1667] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:13:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:13:38.394879 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f556f8b6-02a9-487f-b55c-96204b1c6c4d None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:38.395327 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f556f8b6-02a9-487f-b55c-96204b1c6c4d None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:38.395764 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f556f8b6-02a9-487f-b55c-96204b1c6c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:38.427683 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ea0e25-65a5-4b3c-b167-0505c97110fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:38.428489 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea0e25-65a5-4b3c-b167-0505c97110fe service nova] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:38.429418 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea0e25-65a5-4b3c-b167-0505c97110fe service nova] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:38.429967 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea0e25-65a5-4b3c-b167-0505c97110fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:38.435081 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-71ea0e25-65a5-4b3c-b167-0505c97110fe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:38.445495 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-f556f8b6-02a9-487f-b55c-96204b1c6c4d None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:13:38.447533 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-f556f8b6-02a9-487f-b55c-96204b1c6c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:38.473816 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 834/1668] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:13:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:13:38.475901 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e01ef1-160b-481e-8077-2360529820c7 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:38.476358 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e01ef1-160b-481e-8077-2360529820c7 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:38.476790 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e01ef1-160b-481e-8077-2360529820c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:38.551862 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-26e01ef1-160b-481e-8077-2360529820c7 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:13:38.554530 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-26e01ef1-160b-481e-8077-2360529820c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:38.608710 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 835/1669] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:13:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:13:38.615437 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88441bd-b5d5-4eab-82c1-23765d54a9e5 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:38.615930 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88441bd-b5d5-4eab-82c1-23765d54a9e5 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:38.616417 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88441bd-b5d5-4eab-82c1-23765d54a9e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:38.628388 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ea0e25-65a5-4b3c-b167-0505c97110fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:38.632173 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ea0e25-65a5-4b3c-b167-0505c97110fe service nova] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:38.674815 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 835/1670] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:13:37 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 685 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:13:38.699998 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b4956e-a2e5-48d9-a625-3d138dea3a34 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:38.702604 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b4956e-a2e5-48d9-a625-3d138dea3a34 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:38.703124 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b4956e-a2e5-48d9-a625-3d138dea3a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:38.704546 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-a88441bd-b5d5-4eab-82c1-23765d54a9e5 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:13:38.707356 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-a88441bd-b5d5-4eab-82c1-23765d54a9e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:38.749516 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 836/1671] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:13:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:13:38.751829 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c64acde-367e-427c-b40a-a2f86113feb2 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:38.752286 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c64acde-367e-427c-b40a-a2f86113feb2 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:38.755082 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c64acde-367e-427c-b40a-a2f86113feb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:38.778662 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-73b4956e-a2e5-48d9-a625-3d138dea3a34 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:13:38.778662 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-73b4956e-a2e5-48d9-a625-3d138dea3a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:38.802127 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 836/1672] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:13:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:13:38.802127 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b4956e-a2e5-48d9-a625-3d138dea3a34 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:38.824348 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b4956e-a2e5-48d9-a625-3d138dea3a34 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:38.833521 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-3c64acde-367e-427c-b40a-a2f86113feb2 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:13:38.846628 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-3c64acde-367e-427c-b40a-a2f86113feb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:38.848652 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5bb7a1-a5aa-4d18-8936-1cbad5501dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:38.859918 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5bb7a1-a5aa-4d18-8936-1cbad5501dc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:38.859918 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5bb7a1-a5aa-4d18-8936-1cbad5501dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:38.859918 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5bb7a1-a5aa-4d18-8936-1cbad5501dc9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:38.859918 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5bb7a1-a5aa-4d18-8936-1cbad5501dc9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:13:38.859918 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5bb7a1-a5aa-4d18-8936-1cbad5501dc9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:38.859918 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5bb7a1-a5aa-4d18-8936-1cbad5501dc9 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:38.897509 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 837/1673] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:13:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:13:38.899367 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c64acde-367e-427c-b40a-a2f86113feb2 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:38.902263 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 837/1674] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:13:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:13:38.903234 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5bb7a1-a5aa-4d18-8936-1cbad5501dc9 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:38.945502 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5bb7a1-a5aa-4d18-8936-1cbad5501dc9 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:38.950814 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c64acde-367e-427c-b40a-a2f86113feb2 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:38.972084 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0906ad-7b44-478f-aa5c-fa51bb49f9e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:38.972084 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0906ad-7b44-478f-aa5c-fa51bb49f9e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:38.972084 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0906ad-7b44-478f-aa5c-fa51bb49f9e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:38.972084 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0906ad-7b44-478f-aa5c-fa51bb49f9e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:38.973461 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0906ad-7b44-478f-aa5c-fa51bb49f9e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:13:38.973461 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0906ad-7b44-478f-aa5c-fa51bb49f9e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:38.973461 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0906ad-7b44-478f-aa5c-fa51bb49f9e7 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:38.978063 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bcab14-4a39-444b-a23f-4160db55e0d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:38.978953 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bcab14-4a39-444b-a23f-4160db55e0d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:38.979430 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bcab14-4a39-444b-a23f-4160db55e0d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:38.979976 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bcab14-4a39-444b-a23f-4160db55e0d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:38.981055 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bcab14-4a39-444b-a23f-4160db55e0d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:13:38.981543 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bcab14-4a39-444b-a23f-4160db55e0d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:38.992724 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bcab14-4a39-444b-a23f-4160db55e0d0 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:39.028976 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 838/1675] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:13:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:13:39.033701 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0906ad-7b44-478f-aa5c-fa51bb49f9e7 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:39.065983 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 838/1676] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:13:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:13:39.068020 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bcab14-4a39-444b-a23f-4160db55e0d0 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:39.080860 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0906ad-7b44-478f-aa5c-fa51bb49f9e7 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:39.095228 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bcab14-4a39-444b-a23f-4160db55e0d0 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:39.107823 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed130723-ba2f-4f74-a585-edefead48e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:39.108638 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed130723-ba2f-4f74-a585-edefead48e77 admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:39.109014 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed130723-ba2f-4f74-a585-edefead48e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:39.109417 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed130723-ba2f-4f74-a585-edefead48e77 admin admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:39.110191 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed130723-ba2f-4f74-a585-edefead48e77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:39.112283 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed130723-ba2f-4f74-a585-edefead48e77 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:39.113151 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6532aa-374c-414d-aeca-7d4896b360aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:39.113964 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6532aa-374c-414d-aeca-7d4896b360aa admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:39.114413 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6532aa-374c-414d-aeca-7d4896b360aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:39.114838 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6532aa-374c-414d-aeca-7d4896b360aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:39.116499 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6532aa-374c-414d-aeca-7d4896b360aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:39.121963 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6532aa-374c-414d-aeca-7d4896b360aa admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:39.182079 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 839/1677] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:13:39 2022] POST /identity/v3/projects => generated 391 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:13:39.183307 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed130723-ba2f-4f74-a585-edefead48e77 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:39.288402 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 839/1678] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:13:39 2022] POST /identity/v3/projects => generated 377 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:13:39.295990 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6532aa-374c-414d-aeca-7d4896b360aa admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:39.444720 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:13:39.515802 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed130723-ba2f-4f74-a585-edefead48e77 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:39.641754 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6532aa-374c-414d-aeca-7d4896b360aa admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:39.661948 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b402713-7311-4b33-801c-a742646c97f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:39.662616 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b402713-7311-4b33-801c-a742646c97f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:39.662893 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b402713-7311-4b33-801c-a742646c97f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:39.663176 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b402713-7311-4b33-801c-a742646c97f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:39.664391 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b402713-7311-4b33-801c-a742646c97f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:39.673651 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b402713-7311-4b33-801c-a742646c97f9 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:39.757820 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 840/1679] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:13:39 2022] POST /identity/v3/users => generated 296 bytes in 468 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:13:39.758950 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b402713-7311-4b33-801c-a742646c97f9 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:39.760740 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355045b1-342f-4b4b-8b59-c96576b2f738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:39.768014 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355045b1-342f-4b4b-8b59-c96576b2f738 admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:39.770634 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355045b1-342f-4b4b-8b59-c96576b2f738 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:39.771680 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355045b1-342f-4b4b-8b59-c96576b2f738 admin admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:39.773424 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355045b1-342f-4b4b-8b59-c96576b2f738 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:39.783860 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355045b1-342f-4b4b-8b59-c96576b2f738 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:39.793013 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b402713-7311-4b33-801c-a742646c97f9 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:39.814737 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f325b628-3747-49c2-b33b-69ed3d2d15cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:39.815378 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f325b628-3747-49c2-b33b-69ed3d2d15cd admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:39.815665 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f325b628-3747-49c2-b33b-69ed3d2d15cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:39.815982 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f325b628-3747-49c2-b33b-69ed3d2d15cd admin admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:39.816619 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f325b628-3747-49c2-b33b-69ed3d2d15cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:39.818670 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f325b628-3747-49c2-b33b-69ed3d2d15cd admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:39.879892 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 841/1680] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:13:39 2022] POST /identity/v3/users => generated 289 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:13:39.880954 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f325b628-3747-49c2-b33b-69ed3d2d15cd admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:39.904613 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f325b628-3747-49c2-b33b-69ed3d2d15cd admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:39.924165 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd29600-31d2-4e59-a828-bb385e5b4ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:39.924998 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd29600-31d2-4e59-a828-bb385e5b4ba8 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:39.925344 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd29600-31d2-4e59-a828-bb385e5b4ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:39.925875 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd29600-31d2-4e59-a828-bb385e5b4ba8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:39.927977 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd29600-31d2-4e59-a828-bb385e5b4ba8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:39.929911 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd29600-31d2-4e59-a828-bb385e5b4ba8 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:39.956987 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 840/1681] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:13:39 2022] POST /identity/v3/projects => generated 363 bytes in 774 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:13:39.959044 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355045b1-342f-4b4b-8b59-c96576b2f738 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:39.980330 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 842/1682] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:13:39 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:13:39.981338 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd29600-31d2-4e59-a828-bb385e5b4ba8 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:40.020143 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd29600-31d2-4e59-a828-bb385e5b4ba8 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:40.108952 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:13:40.138152 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355045b1-342f-4b4b-8b59-c96576b2f738 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:40.149478 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74851813-39c4-40c4-b97d-6a51bfab8499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:40.150900 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74851813-39c4-40c4-b97d-6a51bfab8499 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:40.152084 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74851813-39c4-40c4-b97d-6a51bfab8499 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:40.155588 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74851813-39c4-40c4-b97d-6a51bfab8499 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:40.155995 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f085b8d8-8ab1-4832-a983-ec76e947783d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:40.156969 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74851813-39c4-40c4-b97d-6a51bfab8499 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:40.157278 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f085b8d8-8ab1-4832-a983-ec76e947783d admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:40.157566 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f085b8d8-8ab1-4832-a983-ec76e947783d admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:40.157858 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f085b8d8-8ab1-4832-a983-ec76e947783d admin admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:40.158597 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f085b8d8-8ab1-4832-a983-ec76e947783d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:40.160649 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f085b8d8-8ab1-4832-a983-ec76e947783d admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:40.161231 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74851813-39c4-40c4-b97d-6a51bfab8499 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:40.188940 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 841/1683] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:13:39 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:13:40.191455 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74851813-39c4-40c4-b97d-6a51bfab8499 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:40.215786 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74851813-39c4-40c4-b97d-6a51bfab8499 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:40.219902 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 843/1684] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:13:39 2022] POST /identity/v3/users => generated 282 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:13:40.220548 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f085b8d8-8ab1-4832-a983-ec76e947783d admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:40.235520 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03656ac5-0c08-4b0e-bb2f-1455dbf8ecc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:40.236214 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03656ac5-0c08-4b0e-bb2f-1455dbf8ecc5 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:40.236658 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03656ac5-0c08-4b0e-bb2f-1455dbf8ecc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:40.237042 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03656ac5-0c08-4b0e-bb2f-1455dbf8ecc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:40.237818 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03656ac5-0c08-4b0e-bb2f-1455dbf8ecc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:40.240275 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03656ac5-0c08-4b0e-bb2f-1455dbf8ecc5 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:40.255088 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f085b8d8-8ab1-4832-a983-ec76e947783d admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:40.265460 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6edb6e3-010c-4427-9677-2b5ba3a0bf06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:40.266642 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 842/1685] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:13:40.267300 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6edb6e3-010c-4427-9677-2b5ba3a0bf06 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:40.268022 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6edb6e3-010c-4427-9677-2b5ba3a0bf06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:40.268524 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03656ac5-0c08-4b0e-bb2f-1455dbf8ecc5 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:40.268931 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6edb6e3-010c-4427-9677-2b5ba3a0bf06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:40.271427 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6edb6e3-010c-4427-9677-2b5ba3a0bf06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:40.275902 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6edb6e3-010c-4427-9677-2b5ba3a0bf06 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:40.294730 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03656ac5-0c08-4b0e-bb2f-1455dbf8ecc5 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:40.309125 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38477059-b1fe-4851-b7d4-36dffc015b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:40.310088 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38477059-b1fe-4851-b7d4-36dffc015b6e admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:40.310595 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38477059-b1fe-4851-b7d4-36dffc015b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:40.311215 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38477059-b1fe-4851-b7d4-36dffc015b6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:40.311985 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38477059-b1fe-4851-b7d4-36dffc015b6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:40.313997 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38477059-b1fe-4851-b7d4-36dffc015b6e admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:40.318029 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 844/1686] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:13:40.328387 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6edb6e3-010c-4427-9677-2b5ba3a0bf06 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:40.337911 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 843/1687] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:13:40.339093 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38477059-b1fe-4851-b7d4-36dffc015b6e admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:40.362749 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38477059-b1fe-4851-b7d4-36dffc015b6e admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:40.372436 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6edb6e3-010c-4427-9677-2b5ba3a0bf06 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:40.378837 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b7a410-1de4-473c-9b37-1f6a0dff70bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:40.379730 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b7a410-1de4-473c-9b37-1f6a0dff70bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:40.380154 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b7a410-1de4-473c-9b37-1f6a0dff70bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:40.380529 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b7a410-1de4-473c-9b37-1f6a0dff70bf admin admin] PATH_INFO: `/v3/projects/40f231bdad854961a375ce7323ee6255/users/87108684290945b3abf62784a822fff8/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:40.393900 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5127c170-2bdd-4ffb-8a83-8cd931e2c57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:40.395066 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127c170-2bdd-4ffb-8a83-8cd931e2c57d admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:40.395550 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127c170-2bdd-4ffb-8a83-8cd931e2c57d admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:40.396032 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127c170-2bdd-4ffb-8a83-8cd931e2c57d admin admin] PATH_INFO: `/v3/projects/2b693f20a7834354a7b4988012a30b99/users/739e63f410f94e2f9f59d1705c703fc5/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:40.417729 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b7a410-1de4-473c-9b37-1f6a0dff70bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=40f231bdad854961a375ce7323ee6255, user_id=87108684290945b3abf62784a822fff8, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:40.421063 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b7a410-1de4-473c-9b37-1f6a0dff70bf admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:40.449446 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 844/1688] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:13:40 2022] PUT /identity/v3/projects/40f231bdad854961a375ce7323ee6255/users/87108684290945b3abf62784a822fff8/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:13:40.451681 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b7a410-1de4-473c-9b37-1f6a0dff70bf admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:40.457866 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5127c170-2bdd-4ffb-8a83-8cd931e2c57d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b693f20a7834354a7b4988012a30b99, user_id=739e63f410f94e2f9f59d1705c703fc5, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:40.468764 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5127c170-2bdd-4ffb-8a83-8cd931e2c57d admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:40.510884 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:13:40.534090 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 845/1689] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:13:40 2022] PUT /identity/v3/projects/2b693f20a7834354a7b4988012a30b99/users/739e63f410f94e2f9f59d1705c703fc5/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:13:40.540354 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce1ef0-2678-43cf-839e-8a96064267b4 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:40.543192 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce1ef0-2678-43cf-839e-8a96064267b4 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:40.544098 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce1ef0-2678-43cf-839e-8a96064267b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:40.596791 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b7a410-1de4-473c-9b37-1f6a0dff70bf admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:40.658870 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-cfce1ef0-2678-43cf-839e-8a96064267b4 None None] MFA Rules not processed for user `87108684290945b3abf62784a822fff8`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:13:40.660545 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-cfce1ef0-2678-43cf-839e-8a96064267b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:40.829871 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6cb641-b76f-4450-820e-3f02c84e0094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:40.832448 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6cb641-b76f-4450-820e-3f02c84e0094 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:40.833164 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6cb641-b76f-4450-820e-3f02c84e0094 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:40.833585 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6cb641-b76f-4450-820e-3f02c84e0094 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:40.834446 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6cb641-b76f-4450-820e-3f02c84e0094 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:40.840534 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6cb641-b76f-4450-820e-3f02c84e0094 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:40.864740 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 846/1690] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:13:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:13:40.870140 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 845/1691] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 417 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:13:40.871191 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6cb641-b76f-4450-820e-3f02c84e0094 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:40.871317 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de44e90-b4d0-4a13-9d56-4aa0197d17fc None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:40.871317 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de44e90-b4d0-4a13-9d56-4aa0197d17fc None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:40.871569 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de44e90-b4d0-4a13-9d56-4aa0197d17fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:40.893361 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6cb641-b76f-4450-820e-3f02c84e0094 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:40.906094 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39fb589-7c38-4c94-9c50-a4bc5253db12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:40.908463 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39fb589-7c38-4c94-9c50-a4bc5253db12 admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:40.908892 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39fb589-7c38-4c94-9c50-a4bc5253db12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:40.909277 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39fb589-7c38-4c94-9c50-a4bc5253db12 admin admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:40.910081 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39fb589-7c38-4c94-9c50-a4bc5253db12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:40.913551 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39fb589-7c38-4c94-9c50-a4bc5253db12 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:40.946493 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-2de44e90-b4d0-4a13-9d56-4aa0197d17fc None None] MFA Rules not processed for user `739e63f410f94e2f9f59d1705c703fc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:13:40.948137 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-2de44e90-b4d0-4a13-9d56-4aa0197d17fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:40.989209 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 846/1692] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:13:40 2022] POST /identity/v3/projects => generated 377 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:13:40.989209 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39fb589-7c38-4c94-9c50-a4bc5253db12 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:41.118298 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 847/1693] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:13:40 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:13:41.119380 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de44e90-b4d0-4a13-9d56-4aa0197d17fc None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:41.147475 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de44e90-b4d0-4a13-9d56-4aa0197d17fc None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:41.177379 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:13:41.202238 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39fb589-7c38-4c94-9c50-a4bc5253db12 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:41.233150 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae40db-eb3b-48c0-8c6a-e676c87935bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:41.246670 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae40db-eb3b-48c0-8c6a-e676c87935bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:41.247095 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae40db-eb3b-48c0-8c6a-e676c87935bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:41.247509 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae40db-eb3b-48c0-8c6a-e676c87935bc admin admin] PATH_INFO: `/v3/projects/3fca28b39fc84e509f256920c800d468/users/e28246a7df3d4fbd9e7aa98fb293d807/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:41.293415 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb4ea8a-1233-475d-9108-f29b01a51d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:41.294070 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4ea8a-1233-475d-9108-f29b01a51d29 admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:41.294353 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4ea8a-1233-475d-9108-f29b01a51d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:41.294681 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4ea8a-1233-475d-9108-f29b01a51d29 admin admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:41.295605 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb4ea8a-1233-475d-9108-f29b01a51d29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:41.297724 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb4ea8a-1233-475d-9108-f29b01a51d29 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:41.305118 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ae40db-eb3b-48c0-8c6a-e676c87935bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fca28b39fc84e509f256920c800d468, user_id=e28246a7df3d4fbd9e7aa98fb293d807, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:41.315298 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ae40db-eb3b-48c0-8c6a-e676c87935bc admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:41.348388 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 848/1694] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:13:41 2022] POST /identity/v3/users => generated 295 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:13:41.349304 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb4ea8a-1233-475d-9108-f29b01a51d29 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:41.376234 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 847/1695] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:13:40 2022] PUT /identity/v3/projects/3fca28b39fc84e509f256920c800d468/users/e28246a7df3d4fbd9e7aa98fb293d807/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:13:41.379721 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae40db-eb3b-48c0-8c6a-e676c87935bc admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:41.389451 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb4ea8a-1233-475d-9108-f29b01a51d29 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:41.517167 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:13:41.568590 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21568d8-f1ea-41de-bf19-8cb67131a980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:41.569647 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21568d8-f1ea-41de-bf19-8cb67131a980 admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:41.570777 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21568d8-f1ea-41de-bf19-8cb67131a980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:41.572161 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21568d8-f1ea-41de-bf19-8cb67131a980 admin admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:41.574330 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21568d8-f1ea-41de-bf19-8cb67131a980 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:41.582295 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21568d8-f1ea-41de-bf19-8cb67131a980 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:41.597473 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae40db-eb3b-48c0-8c6a-e676c87935bc admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:41.609636 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bce94ad-7918-4e4b-a99b-a775b7f4e608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:41.610289 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce94ad-7918-4e4b-a99b-a775b7f4e608 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:41.610600 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce94ad-7918-4e4b-a99b-a775b7f4e608 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:41.611050 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce94ad-7918-4e4b-a99b-a775b7f4e608 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:41.614489 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bce94ad-7918-4e4b-a99b-a775b7f4e608 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:41.616271 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bce94ad-7918-4e4b-a99b-a775b7f4e608 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:41.650084 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 848/1696] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:13:41 2022] GET /identity/v3/roles => generated 1420 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:13:41.650084 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ca082f-6b7b-49e3-9fd3-c470603a80e1 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:41.650084 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ca082f-6b7b-49e3-9fd3-c470603a80e1 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:41.650084 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ca082f-6b7b-49e3-9fd3-c470603a80e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:41.691684 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 849/1697] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:13:41 2022] POST /identity/v3/projects => generated 393 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:13:41.693272 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21568d8-f1ea-41de-bf19-8cb67131a980 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:41.736583 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-02ca082f-6b7b-49e3-9fd3-c470603a80e1 None None] MFA Rules not processed for user `e28246a7df3d4fbd9e7aa98fb293d807`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:13:41.737908 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-02ca082f-6b7b-49e3-9fd3-c470603a80e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:41.976798 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21568d8-f1ea-41de-bf19-8cb67131a980 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:42.002177 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a54f7e-e542-4d1f-b8c5-a256e98eb8f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:42.007362 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 849/1698] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:13:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:13:42.007362 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ca082f-6b7b-49e3-9fd3-c470603a80e1 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:42.009283 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a54f7e-e542-4d1f-b8c5-a256e98eb8f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:42.009992 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a54f7e-e542-4d1f-b8c5-a256e98eb8f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:42.011354 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a54f7e-e542-4d1f-b8c5-a256e98eb8f2 admin admin] PATH_INFO: `/v3/projects/ba3247c9fefb4027ac363189468900dd/users/f01399cce6c742cdb28e14ab580c3566/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:42.043151 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ca082f-6b7b-49e3-9fd3-c470603a80e1 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:42.060027 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe74f32-a4f6-4203-b2b4-99fc983a1f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:42.060684 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe74f32-a4f6-4203-b2b4-99fc983a1f90 admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:42.060968 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe74f32-a4f6-4203-b2b4-99fc983a1f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:42.061243 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe74f32-a4f6-4203-b2b4-99fc983a1f90 admin admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:42.065147 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe74f32-a4f6-4203-b2b4-99fc983a1f90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:42.069409 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe74f32-a4f6-4203-b2b4-99fc983a1f90 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:42.081957 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a54f7e-e542-4d1f-b8c5-a256e98eb8f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba3247c9fefb4027ac363189468900dd, user_id=f01399cce6c742cdb28e14ab580c3566, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:42.105304 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a54f7e-e542-4d1f-b8c5-a256e98eb8f2 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:42.125918 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 850/1699] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:13:42 2022] POST /identity/v3/users => generated 303 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:13:42.126870 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe74f32-a4f6-4203-b2b4-99fc983a1f90 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:42.159102 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 850/1700] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:13:41 2022] PUT /identity/v3/projects/ba3247c9fefb4027ac363189468900dd/users/f01399cce6c742cdb28e14ab580c3566/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:13:42.159746 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a54f7e-e542-4d1f-b8c5-a256e98eb8f2 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:42.168911 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe74f32-a4f6-4203-b2b4-99fc983a1f90 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:42.195625 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:13:42.348755 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205a60b5-06d3-4916-9b1e-14cb1a7950a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:42.349406 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205a60b5-06d3-4916-9b1e-14cb1a7950a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:42.349689 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205a60b5-06d3-4916-9b1e-14cb1a7950a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:42.349966 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205a60b5-06d3-4916-9b1e-14cb1a7950a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:42.350917 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205a60b5-06d3-4916-9b1e-14cb1a7950a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:42.356590 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205a60b5-06d3-4916-9b1e-14cb1a7950a5 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:42.370775 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a54f7e-e542-4d1f-b8c5-a256e98eb8f2 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:42.381930 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e41d802-7b3c-4dd0-8444-b5db0dd2b339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:42.382601 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e41d802-7b3c-4dd0-8444-b5db0dd2b339 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:42.382883 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e41d802-7b3c-4dd0-8444-b5db0dd2b339 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:42.383176 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e41d802-7b3c-4dd0-8444-b5db0dd2b339 admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:42.384366 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e41d802-7b3c-4dd0-8444-b5db0dd2b339 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:42.387714 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e41d802-7b3c-4dd0-8444-b5db0dd2b339 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:42.416954 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 851/1701] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:13:42 2022] GET /identity/v3/roles => generated 1420 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:13:42.419221 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e41d802-7b3c-4dd0-8444-b5db0dd2b339 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:42.438632 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 851/1702] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:13:42 2022] POST /identity/v3/projects => generated 363 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:13:42.440166 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2684620-68b8-4e16-850a-cecd3a18ddeb None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:42.440457 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2684620-68b8-4e16-850a-cecd3a18ddeb None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:42.440719 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2684620-68b8-4e16-850a-cecd3a18ddeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:42.478146 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e41d802-7b3c-4dd0-8444-b5db0dd2b339 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:42.539165 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-f2684620-68b8-4e16-850a-cecd3a18ddeb None None] MFA Rules not processed for user `428a95af27d2468bb39391ca3eefea40`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:13:42.549591 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-f2684620-68b8-4e16-850a-cecd3a18ddeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:42.592508 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:13:42.770727 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b0a2e-646b-4192-9b52-f33065ad7535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:42.771866 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b0a2e-646b-4192-9b52-f33065ad7535 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:42.772435 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b0a2e-646b-4192-9b52-f33065ad7535 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:42.784642 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b0a2e-646b-4192-9b52-f33065ad7535 admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:42.789098 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5b0a2e-646b-4192-9b52-f33065ad7535 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:42.793437 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5b0a2e-646b-4192-9b52-f33065ad7535 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:42.838329 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 852/1703] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:13:42 2022] GET /identity/v3/roles => generated 1420 bytes in 420 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:13:42.839638 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b0a2e-646b-4192-9b52-f33065ad7535 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:42.869917 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b0a2e-646b-4192-9b52-f33065ad7535 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:42.878568 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 852/1704] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:13:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 438 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:13:42.883022 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2684620-68b8-4e16-850a-cecd3a18ddeb None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:42.891109 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9689feb2-6239-4cd3-bca5-592efff851bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:42.892116 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9689feb2-6239-4cd3-bca5-592efff851bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:42.892664 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9689feb2-6239-4cd3-bca5-592efff851bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:42.893100 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9689feb2-6239-4cd3-bca5-592efff851bb admin admin] PATH_INFO: `/v3/projects/d12f082dcbd34ea781bd046fa64664ec/users/75878aa281c64627959206b564c7ead8/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:42.918477 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2684620-68b8-4e16-850a-cecd3a18ddeb None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:42.926258 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9689feb2-6239-4cd3-bca5-592efff851bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d12f082dcbd34ea781bd046fa64664ec, user_id=75878aa281c64627959206b564c7ead8, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:42.934600 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9689feb2-6239-4cd3-bca5-592efff851bb admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:42.935713 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6ab245-6365-472d-8537-630aec55c0c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:42.940537 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6ab245-6365-472d-8537-630aec55c0c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:42.940873 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6ab245-6365-472d-8537-630aec55c0c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:42.941156 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6ab245-6365-472d-8537-630aec55c0c7 admin admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:42.941866 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6ab245-6365-472d-8537-630aec55c0c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:42.954674 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6ab245-6365-472d-8537-630aec55c0c7 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:42.964237 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 853/1705] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:13:42 2022] PUT /identity/v3/projects/d12f082dcbd34ea781bd046fa64664ec/users/75878aa281c64627959206b564c7ead8/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:13:42.965589 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea14271-12a8-441d-8479-dddcb8fc5d6c None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:42.965881 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea14271-12a8-441d-8479-dddcb8fc5d6c None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:42.966176 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea14271-12a8-441d-8479-dddcb8fc5d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:43.016759 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 853/1706] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:13:42 2022] POST /identity/v3/users => generated 288 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:13:43.017661 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6ab245-6365-472d-8537-630aec55c0c7 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:43.039511 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-9ea14271-12a8-441d-8479-dddcb8fc5d6c None None] MFA Rules not processed for user `754c1b5cb47746cf93c52d6dc0b8fa01`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:13:43.041351 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-9ea14271-12a8-441d-8479-dddcb8fc5d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:43.055677 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6ab245-6365-472d-8537-630aec55c0c7 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:43.224938 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c05ad33-f6c5-4f20-9baf-95e7479be00a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:43.228373 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c05ad33-f6c5-4f20-9baf-95e7479be00a admin admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:43.232473 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c05ad33-f6c5-4f20-9baf-95e7479be00a admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:43.233159 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c05ad33-f6c5-4f20-9baf-95e7479be00a admin admin] PATH_INFO: `/v3/projects/ba3247c9fefb4027ac363189468900dd/users/f01399cce6c742cdb28e14ab580c3566/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:43.237780 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c05ad33-f6c5-4f20-9baf-95e7479be00a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba3247c9fefb4027ac363189468900dd, user_id=f01399cce6c742cdb28e14ab580c3566, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:43.240727 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c05ad33-f6c5-4f20-9baf-95e7479be00a admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:43.250330 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 854/1707] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:13:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:13:43.253775 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceee38b-ddbf-4aa0-804b-0c518240705f None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:43.256296 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceee38b-ddbf-4aa0-804b-0c518240705f None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:43.256629 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceee38b-ddbf-4aa0-804b-0c518240705f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:43.287713 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 854/1708] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:13:43 2022] PUT /identity/v3/projects/ba3247c9fefb4027ac363189468900dd/users/f01399cce6c742cdb28e14ab580c3566/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:13:43.287713 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c05ad33-f6c5-4f20-9baf-95e7479be00a admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:43.307661 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:13:43.326203 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-9ceee38b-ddbf-4aa0-804b-0c518240705f None None] MFA Rules not processed for user `428a95af27d2468bb39391ca3eefea40`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:13:43.328500 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-9ceee38b-ddbf-4aa0-804b-0c518240705f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:43.483407 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c05ad33-f6c5-4f20-9baf-95e7479be00a admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:43.488307 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 855/1709] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:13:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:13:43.489243 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceee38b-ddbf-4aa0-804b-0c518240705f None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:43.494645 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cecf99-d70a-40aa-86fd-8e75d357f5c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:43.498548 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cecf99-d70a-40aa-86fd-8e75d357f5c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:43.499185 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cecf99-d70a-40aa-86fd-8e75d357f5c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:43.499671 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cecf99-d70a-40aa-86fd-8e75d357f5c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:43.501590 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cecf99-d70a-40aa-86fd-8e75d357f5c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:43.506094 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cecf99-d70a-40aa-86fd-8e75d357f5c0 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:43.522703 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceee38b-ddbf-4aa0-804b-0c518240705f None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:43.531020 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 855/1710] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:13:43 2022] GET /identity/v3/roles => generated 1420 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:13:43.533213 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b710b3-1113-4d96-87e5-1ef9a99243df None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:43.533811 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b710b3-1113-4d96-87e5-1ef9a99243df None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:43.534315 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b710b3-1113-4d96-87e5-1ef9a99243df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:43.546231 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3840bb-8c91-4823-b8b0-74d922b31bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:43.546906 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3840bb-8c91-4823-b8b0-74d922b31bd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:43.547185 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3840bb-8c91-4823-b8b0-74d922b31bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:43.547456 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3840bb-8c91-4823-b8b0-74d922b31bd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:43.548775 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3840bb-8c91-4823-b8b0-74d922b31bd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:43.563478 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3840bb-8c91-4823-b8b0-74d922b31bd3 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:43.603970 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 856/1711] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:13:43 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:13:43.605497 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c69e6ef-5f30-4cd2-a67b-80eb2fc5c04a None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:43.605790 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c69e6ef-5f30-4cd2-a67b-80eb2fc5c04a None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:43.606074 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c69e6ef-5f30-4cd2-a67b-80eb2fc5c04a None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:43.607522 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 857/1712] 10.222.0.57 () {54 vars in 965 bytes} [Mon Sep 19 16:13:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:13:43.608327 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c69e6ef-5f30-4cd2-a67b-80eb2fc5c04a None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:43.614308 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-a9b710b3-1113-4d96-87e5-1ef9a99243df None None] MFA Rules not processed for user `f01399cce6c742cdb28e14ab580c3566`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:13:43.616388 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-a9b710b3-1113-4d96-87e5-1ef9a99243df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:43.627193 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-5c69e6ef-5f30-4cd2-a67b-80eb2fc5c04a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:43.670390 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:13:43.916263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c69e6ef-5f30-4cd2-a67b-80eb2fc5c04a None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:43.942262 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81723e85-8f31-40d0-a5d8-83c3f45bd2cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:43.942965 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81723e85-8f31-40d0-a5d8-83c3f45bd2cb service nova] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:43.943255 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81723e85-8f31-40d0-a5d8-83c3f45bd2cb service nova] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:43.943534 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81723e85-8f31-40d0-a5d8-83c3f45bd2cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:43.994690 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81723e85-8f31-40d0-a5d8-83c3f45bd2cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:43.996376 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81723e85-8f31-40d0-a5d8-83c3f45bd2cb service nova] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:44.020464 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 856/1713] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:13:43 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 488 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:13:44.020938 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b710b3-1113-4d96-87e5-1ef9a99243df None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:44.061401 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 858/1714] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:13:43 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:13:44.062602 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81723e85-8f31-40d0-a5d8-83c3f45bd2cb service nova] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:44.084571 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b710b3-1113-4d96-87e5-1ef9a99243df None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:44.090164 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81723e85-8f31-40d0-a5d8-83c3f45bd2cb service nova] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:44.099857 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76a1a4d-8a57-4a03-8999-10e46bc05831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:44.100889 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97bee77-7037-4147-b11b-badd61d8edca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:44.102889 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97bee77-7037-4147-b11b-badd61d8edca admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:44.102889 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97bee77-7037-4147-b11b-badd61d8edca admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:44.102889 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97bee77-7037-4147-b11b-badd61d8edca admin admin] PATH_INFO: `/v3/projects/dc9d27fbfcc347cc9960a4802fb7e8b8/users/1b27b454d2d944e29aee8329564296c7/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:44.103378 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76a1a4d-8a57-4a03-8999-10e46bc05831 admin admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:44.103727 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76a1a4d-8a57-4a03-8999-10e46bc05831 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:44.104103 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76a1a4d-8a57-4a03-8999-10e46bc05831 admin admin] PATH_INFO: `/v3/projects/d12f082dcbd34ea781bd046fa64664ec/users/75878aa281c64627959206b564c7ead8/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:44.111327 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76a1a4d-8a57-4a03-8999-10e46bc05831 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d12f082dcbd34ea781bd046fa64664ec, user_id=75878aa281c64627959206b564c7ead8, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:44.120284 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76a1a4d-8a57-4a03-8999-10e46bc05831 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:44.133410 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97bee77-7037-4147-b11b-badd61d8edca admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc9d27fbfcc347cc9960a4802fb7e8b8, user_id=1b27b454d2d944e29aee8329564296c7, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:44.136310 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97bee77-7037-4147-b11b-badd61d8edca admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:44.170543 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 857/1715] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:13:44 2022] PUT /identity/v3/projects/d12f082dcbd34ea781bd046fa64664ec/users/75878aa281c64627959206b564c7ead8/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:13:44.170543 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f21754b-e1f8-4d66-8373-e2514f5df69f None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:44.170543 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f21754b-e1f8-4d66-8373-e2514f5df69f None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:44.170543 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f21754b-e1f8-4d66-8373-e2514f5df69f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:44.182758 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 859/1716] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:13:44 2022] PUT /identity/v3/projects/dc9d27fbfcc347cc9960a4802fb7e8b8/users/1b27b454d2d944e29aee8329564296c7/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:13:44.183812 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97bee77-7037-4147-b11b-badd61d8edca admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:44.192636 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-b97bee77-7037-4147-b11b-badd61d8edca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:44.231870 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-6f21754b-e1f8-4d66-8373-e2514f5df69f None None] MFA Rules not processed for user `1ecd01cc72834b72b5ff2c4b47249737`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:13:44.233280 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-6f21754b-e1f8-4d66-8373-e2514f5df69f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:44.386343 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97bee77-7037-4147-b11b-badd61d8edca admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:44.395979 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e88a4a-8f4f-4b62-8ae5-5cafcbe566c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:44.400384 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e88a4a-8f4f-4b62-8ae5-5cafcbe566c9 service cinder] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:44.400676 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e88a4a-8f4f-4b62-8ae5-5cafcbe566c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:44.400961 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e88a4a-8f4f-4b62-8ae5-5cafcbe566c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:44.436828 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:13:44.515556 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 858/1717] 10.222.0.57 () {58 vars in 1062 bytes} [Mon Sep 19 16:13:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:13:44.518197 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a4ebb9-424d-4320-8805-ddf68908956d None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:44.520612 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a4ebb9-424d-4320-8805-ddf68908956d None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:44.524669 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a4ebb9-424d-4320-8805-ddf68908956d None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:44.542957 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 859/1718] 10.222.0.57 () {54 vars in 912 bytes} [Mon Sep 19 16:13:44 2022] GET /identity => generated 270 bytes in 26 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:13:44.545725 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c26b4-bb1f-4ec8-9d91-6a287b3e86c6 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:44.546469 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c26b4-bb1f-4ec8-9d91-6a287b3e86c6 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:44.547338 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c26b4-bb1f-4ec8-9d91-6a287b3e86c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:44.563920 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e88a4a-8f4f-4b62-8ae5-5cafcbe566c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:44.565565 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e88a4a-8f4f-4b62-8ae5-5cafcbe566c9 service cinder] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:44.604605 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 860/1719] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:13:44 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:13:44.606287 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e88a4a-8f4f-4b62-8ae5-5cafcbe566c9 service cinder] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:44.662831 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-695c26b4-bb1f-4ec8-9d91-6a287b3e86c6 None None] MFA Rules not processed for user `75878aa281c64627959206b564c7ead8`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:13:44.667839 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-695c26b4-bb1f-4ec8-9d91-6a287b3e86c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:44.798875 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e88a4a-8f4f-4b62-8ae5-5cafcbe566c9 service cinder] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:44.813472 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4586c1-2584-4a69-a02e-e1dcef47c7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:44.814178 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4586c1-2584-4a69-a02e-e1dcef47c7ab admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:44.815575 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4586c1-2584-4a69-a02e-e1dcef47c7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:44.815896 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4586c1-2584-4a69-a02e-e1dcef47c7ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:44.816601 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4586c1-2584-4a69-a02e-e1dcef47c7ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:44.820493 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4586c1-2584-4a69-a02e-e1dcef47c7ab admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:44.872960 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 861/1720] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:13:44 2022] GET /identity/v3/roles => generated 1420 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:13:44.874917 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a19bb8-d799-460d-8fc6-6a9d38e937ae None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:44.875212 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a19bb8-d799-460d-8fc6-6a9d38e937ae None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:44.885228 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a19bb8-d799-460d-8fc6-6a9d38e937ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:44.995788 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-a8a19bb8-d799-460d-8fc6-6a9d38e937ae None None] MFA Rules not processed for user `8cd9c727c0b444cb83497ea4f78e24d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:13:44.999328 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-a8a19bb8-d799-460d-8fc6-6a9d38e937ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:45.129701 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 860/1721] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:13:44 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 584 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:13:45.132190 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695c26b4-bb1f-4ec8-9d91-6a287b3e86c6 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:45.184319 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695c26b4-bb1f-4ec8-9d91-6a287b3e86c6 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:45.203167 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd445641-15b2-423b-b681-f4a06c35871c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:45.203872 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd445641-15b2-423b-b681-f4a06c35871c service nova] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:45.204164 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd445641-15b2-423b-b681-f4a06c35871c service nova] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:45.204437 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd445641-15b2-423b-b681-f4a06c35871c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:45.217858 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 862/1722] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:13:44 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:13:45.219403 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4c8d79-f669-4de9-b92d-2310b189aa4b None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:45.219694 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4c8d79-f669-4de9-b92d-2310b189aa4b None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:45.219974 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4c8d79-f669-4de9-b92d-2310b189aa4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:45.246037 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:13:45.291106 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-8b4c8d79-f669-4de9-b92d-2310b189aa4b None None] MFA Rules not processed for user `1ecd01cc72834b72b5ff2c4b47249737`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:13:45.294712 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-8b4c8d79-f669-4de9-b92d-2310b189aa4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:45.345447 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 863/1723] 10.222.0.57 () {58 vars in 1009 bytes} [Mon Sep 19 16:13:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:13:45.348339 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4c8d79-f669-4de9-b92d-2310b189aa4b None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:45.377994 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd445641-15b2-423b-b681-f4a06c35871c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:45.380534 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd445641-15b2-423b-b681-f4a06c35871c service nova] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:45.390329 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4c8d79-f669-4de9-b92d-2310b189aa4b None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:45.407556 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 861/1724] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:13:45 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:13:45.408583 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd445641-15b2-423b-b681-f4a06c35871c service nova] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:45.412509 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-dd445641-15b2-423b-b681-f4a06c35871c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:45.421217 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5459ba1c-c513-4014-9bf1-a3def55d8c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:45.423320 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5459ba1c-c513-4014-9bf1-a3def55d8c72 admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:45.423607 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5459ba1c-c513-4014-9bf1-a3def55d8c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:45.424019 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5459ba1c-c513-4014-9bf1-a3def55d8c72 admin admin] PATH_INFO: `/v3/projects/dc9d27fbfcc347cc9960a4802fb7e8b8/users/1b27b454d2d944e29aee8329564296c7/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:45.431997 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5459ba1c-c513-4014-9bf1-a3def55d8c72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc9d27fbfcc347cc9960a4802fb7e8b8, user_id=1b27b454d2d944e29aee8329564296c7, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:45.440062 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5459ba1c-c513-4014-9bf1-a3def55d8c72 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:45.440285 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd445641-15b2-423b-b681-f4a06c35871c service nova] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:45.449423 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ef29d0-a02a-4ed1-8b1e-1532c7882ccd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:45.450126 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ef29d0-a02a-4ed1-8b1e-1532c7882ccd service cinder] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:45.450433 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ef29d0-a02a-4ed1-8b1e-1532c7882ccd service cinder] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:45.450729 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ef29d0-a02a-4ed1-8b1e-1532c7882ccd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:45.480561 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 864/1725] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:13:45 2022] PUT /identity/v3/projects/dc9d27fbfcc347cc9960a4802fb7e8b8/users/1b27b454d2d944e29aee8329564296c7/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:13:45.482256 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7915511-22e3-48c9-b3b4-61369ae016a5 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:45.482750 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7915511-22e3-48c9-b3b4-61369ae016a5 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:45.484650 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7915511-22e3-48c9-b3b4-61369ae016a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:45.489979 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ef29d0-a02a-4ed1-8b1e-1532c7882ccd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:45.491814 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ef29d0-a02a-4ed1-8b1e-1532c7882ccd service cinder] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:45.529601 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 862/1726] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:13:45 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:13:45.530643 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ef29d0-a02a-4ed1-8b1e-1532c7882ccd service cinder] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:45.531335 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-c7915511-22e3-48c9-b3b4-61369ae016a5 None None] MFA Rules not processed for user `8cd9c727c0b444cb83497ea4f78e24d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:13:45.533085 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-c7915511-22e3-48c9-b3b4-61369ae016a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:45.539308 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-a4ef29d0-a02a-4ed1-8b1e-1532c7882ccd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:45.600700 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:13:45.870824 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 865/1727] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:13:45 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 389 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:13:45.872323 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3bde2b-291d-4b65-8328-3fd1d2f1b97f None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:45.879288 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3bde2b-291d-4b65-8328-3fd1d2f1b97f None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:45.879288 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3bde2b-291d-4b65-8328-3fd1d2f1b97f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:45.958977 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-ca3bde2b-291d-4b65-8328-3fd1d2f1b97f None None] MFA Rules not processed for user `1b27b454d2d944e29aee8329564296c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:13:45.961302 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ef29d0-a02a-4ed1-8b1e-1532c7882ccd service cinder] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:45.963404 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-ca3bde2b-291d-4b65-8328-3fd1d2f1b97f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:46.223056 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7e1dd0-3d23-4486-9327-d11fc0a8a5ba service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5632ade0cc6f4ae28f72160de68486cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:46.224152 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7e1dd0-3d23-4486-9327-d11fc0a8a5ba service glance] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:46.224630 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7e1dd0-3d23-4486-9327-d11fc0a8a5ba service glance] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:46.225114 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7e1dd0-3d23-4486-9327-d11fc0a8a5ba service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:46.258636 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 866/1728] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:13:45 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 383 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:13:46.259472 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3bde2b-291d-4b65-8328-3fd1d2f1b97f None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:46.613787 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3bde2b-291d-4b65-8328-3fd1d2f1b97f None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:46.625720 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e92b4d8-a79d-495f-9601-f571149e89a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:46.627036 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e92b4d8-a79d-495f-9601-f571149e89a4 service cinder] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:46.627336 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e92b4d8-a79d-495f-9601-f571149e89a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:46.627616 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e92b4d8-a79d-495f-9601-f571149e89a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:46.652143 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:13:46.799473 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e92b4d8-a79d-495f-9601-f571149e89a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:46.801002 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e92b4d8-a79d-495f-9601-f571149e89a4 service cinder] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:46.808542 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7e1dd0-3d23-4486-9327-d11fc0a8a5ba service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:46.817104 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7e1dd0-3d23-4486-9327-d11fc0a8a5ba service glance] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:46.831257 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 867/1729] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:13:46 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 575 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:13:46.841266 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e92b4d8-a79d-495f-9601-f571149e89a4 service cinder] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:46.889362 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 863/1730] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:13:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 1359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:13:46.891940 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e92b4d8-a79d-495f-9601-f571149e89a4 service cinder] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:46.893053 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7e1dd0-3d23-4486-9327-d11fc0a8a5ba service glance] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:46.920672 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251f4472-e534-44ce-91f6-97c313edac6c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:46.921415 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f4472-e534-44ce-91f6-97c313edac6c service nova] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:46.921736 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f4472-e534-44ce-91f6-97c313edac6c service nova] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:46.922061 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f4472-e534-44ce-91f6-97c313edac6c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:46.946516 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:13:46.963906 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7e1dd0-3d23-4486-9327-d11fc0a8a5ba service glance] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:46.966159 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251f4472-e534-44ce-91f6-97c313edac6c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:46.969960 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251f4472-e534-44ce-91f6-97c313edac6c service nova] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:46.989352 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4145765e-0476-4452-9ef9-42a4574b3f1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:46.999156 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4145765e-0476-4452-9ef9-42a4574b3f1e service cinder] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:46.999816 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4145765e-0476-4452-9ef9-42a4574b3f1e service cinder] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:47.004190 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4145765e-0476-4452-9ef9-42a4574b3f1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:47.018171 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 868/1731] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:13:46 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:13:47.118638 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4145765e-0476-4452-9ef9-42a4574b3f1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:47.129036 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4145765e-0476-4452-9ef9-42a4574b3f1e service cinder] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:47.137820 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e0b45-e9fd-4125-a60a-dd6eb120d181 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:47.138109 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e0b45-e9fd-4125-a60a-dd6eb120d181 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:47.138402 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e0b45-e9fd-4125-a60a-dd6eb120d181 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:47.140288 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 869/1732] 10.222.0.57 () {54 vars in 912 bytes} [Mon Sep 19 16:13:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:13:47.153048 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed49fe8d-1869-41f1-a425-6b0e446ab300 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:47.153334 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed49fe8d-1869-41f1-a425-6b0e446ab300 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:47.153619 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed49fe8d-1869-41f1-a425-6b0e446ab300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:47.229401 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-ed49fe8d-1869-41f1-a425-6b0e446ab300 None None] MFA Rules not processed for user `1ecd01cc72834b72b5ff2c4b47249737`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:13:47.230747 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-ed49fe8d-1869-41f1-a425-6b0e446ab300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:47.242434 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 864/1733] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:13:46 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:13:47.253791 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 870/1734] 10.222.0.57 () {58 vars in 1009 bytes} [Mon Sep 19 16:13:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:13:47.267964 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4145765e-0476-4452-9ef9-42a4574b3f1e service cinder] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:47.312460 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4145765e-0476-4452-9ef9-42a4574b3f1e service cinder] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:47.354926 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd50408-9eb6-4887-9a1f-e9b0ec273872 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5632ade0cc6f4ae28f72160de68486cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:47.361578 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd50408-9eb6-4887-9a1f-e9b0ec273872 service glance] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:47.362421 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd50408-9eb6-4887-9a1f-e9b0ec273872 service glance] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:47.363178 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd50408-9eb6-4887-9a1f-e9b0ec273872 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:47.422756 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd50408-9eb6-4887-9a1f-e9b0ec273872 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:47.425213 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd50408-9eb6-4887-9a1f-e9b0ec273872 service glance] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:47.458791 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 865/1735] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:13:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:13:47.637121 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed49fe8d-1869-41f1-a425-6b0e446ab300 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:47.670999 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed49fe8d-1869-41f1-a425-6b0e446ab300 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:47.686426 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a379d99-3feb-4e7c-964a-f4c2acadea27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:47.687267 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a379d99-3feb-4e7c-964a-f4c2acadea27 service cinder] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:47.687691 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a379d99-3feb-4e7c-964a-f4c2acadea27 service cinder] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:47.688092 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a379d99-3feb-4e7c-964a-f4c2acadea27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:47.954241 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a379d99-3feb-4e7c-964a-f4c2acadea27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:47.957961 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a379d99-3feb-4e7c-964a-f4c2acadea27 service cinder] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:48.009646 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 871/1736] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:13:47 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:13:48.654074 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9486d3-c44b-4346-a0e6-c11467b85e5c None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:48.655732 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9486d3-c44b-4346-a0e6-c11467b85e5c None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:48.656479 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9486d3-c44b-4346-a0e6-c11467b85e5c None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:48.659759 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 866/1737] 10.222.0.57 () {54 vars in 912 bytes} [Mon Sep 19 16:13:48 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:13:48.667427 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f358c206-d404-41db-aaba-cb0cb2f73ddf None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:48.668504 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f358c206-d404-41db-aaba-cb0cb2f73ddf None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:48.668971 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f358c206-d404-41db-aaba-cb0cb2f73ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:48.734839 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e251bb-8a68-4b5b-87d3-6569e2901fd1 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:48.735288 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e251bb-8a68-4b5b-87d3-6569e2901fd1 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:48.735663 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e251bb-8a68-4b5b-87d3-6569e2901fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:48.840941 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-e6e251bb-8a68-4b5b-87d3-6569e2901fd1 None None] MFA Rules not processed for user `bd190d8e252342ca9232f7a0c966c715`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:13:48.843608 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-e6e251bb-8a68-4b5b-87d3-6569e2901fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:48.844955 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-f358c206-d404-41db-aaba-cb0cb2f73ddf None None] MFA Rules not processed for user `50355d7b1c6c4f0bbbbc7fe56efb9039`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:13:48.846495 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-f358c206-d404-41db-aaba-cb0cb2f73ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:49.027040 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 872/1738] 10.222.0.57 () {58 vars in 1009 bytes} [Mon Sep 19 16:13:48 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:13:49.031162 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 867/1739] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:13:48 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:13:49.186609 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659ef5e-96db-4e19-ad07-78712f4fdb41 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:49.187056 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659ef5e-96db-4e19-ad07-78712f4fdb41 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:49.187435 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659ef5e-96db-4e19-ad07-78712f4fdb41 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:49.190641 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 873/1740] 10.222.0.57 () {54 vars in 912 bytes} [Mon Sep 19 16:13:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 16:13:49.201958 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e251bb-8a68-4b5b-87d3-6569e2901fd1 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:49.208483 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f6117d-49aa-43b0-a784-a140b6a34a7e None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:49.208928 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f6117d-49aa-43b0-a784-a140b6a34a7e None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:49.209314 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f6117d-49aa-43b0-a784-a140b6a34a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:49.213254 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-e6e251bb-8a68-4b5b-87d3-6569e2901fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:49.253978 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e251bb-8a68-4b5b-87d3-6569e2901fd1 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:49.268074 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24392e58-325b-48de-a418-97dde85e7ea4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:49.269809 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24392e58-325b-48de-a418-97dde85e7ea4 service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:49.270550 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24392e58-325b-48de-a418-97dde85e7ea4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:49.276824 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24392e58-325b-48de-a418-97dde85e7ea4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:49.292977 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-d4f6117d-49aa-43b0-a784-a140b6a34a7e None None] MFA Rules not processed for user `50355d7b1c6c4f0bbbbc7fe56efb9039`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:13:49.295631 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-d4f6117d-49aa-43b0-a784-a140b6a34a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:49.303661 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24392e58-325b-48de-a418-97dde85e7ea4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:49.306058 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24392e58-325b-48de-a418-97dde85e7ea4 service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:49.316965 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 868/1741] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:13:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:13:49.335344 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 874/1742] 10.222.0.57 () {58 vars in 1009 bytes} [Mon Sep 19 16:13:49 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:13:49.366573 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24392e58-325b-48de-a418-97dde85e7ea4 service neutron] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:49.368722 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-24392e58-325b-48de-a418-97dde85e7ea4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:49.394365 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24392e58-325b-48de-a418-97dde85e7ea4 service neutron] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:49.409171 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a5ef52-3503-405b-b6eb-ec16be65f0ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:49.410236 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a5ef52-3503-405b-b6eb-ec16be65f0ad service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:49.410815 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a5ef52-3503-405b-b6eb-ec16be65f0ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:49.411553 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a5ef52-3503-405b-b6eb-ec16be65f0ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:49.424821 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a5ef52-3503-405b-b6eb-ec16be65f0ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:49.427876 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a5ef52-3503-405b-b6eb-ec16be65f0ad service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:49.439204 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 869/1743] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:13:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:13:49.831888 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc0ca2-e4eb-476f-ac0f-927e312babb2 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:49.833733 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc0ca2-e4eb-476f-ac0f-927e312babb2 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:49.834455 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc0ca2-e4eb-476f-ac0f-927e312babb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:49.912142 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-1fbc0ca2-e4eb-476f-ac0f-927e312babb2 None None] MFA Rules not processed for user `fd36089b3c2141218660ab2458376798`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:13:49.923137 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-1fbc0ca2-e4eb-476f-ac0f-927e312babb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:50.143038 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 875/1744] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:13:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:13:50.524359 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a5ef52-3503-405b-b6eb-ec16be65f0ad service neutron] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:50.553808 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a5ef52-3503-405b-b6eb-ec16be65f0ad service neutron] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:50.579082 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61abdeb0-04e7-487a-9e5b-f03275728ee3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:50.589054 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61abdeb0-04e7-487a-9e5b-f03275728ee3 service nova] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:50.589335 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61abdeb0-04e7-487a-9e5b-f03275728ee3 service nova] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:50.592393 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61abdeb0-04e7-487a-9e5b-f03275728ee3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:50.628933 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61abdeb0-04e7-487a-9e5b-f03275728ee3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:50.630692 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61abdeb0-04e7-487a-9e5b-f03275728ee3 service nova] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:50.755126 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 870/1745] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:13:50 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:13:50.834081 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbc0ca2-e4eb-476f-ac0f-927e312babb2 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:50.837394 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-1fbc0ca2-e4eb-476f-ac0f-927e312babb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:50.886671 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:13:50.969087 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e49a6-1ef2-425a-bb49-4ee7be01d59e None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:50.971435 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e49a6-1ef2-425a-bb49-4ee7be01d59e None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:50.972291 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e49a6-1ef2-425a-bb49-4ee7be01d59e None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:50.974526 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 871/1746] 10.222.0.57 () {54 vars in 961 bytes} [Mon Sep 19 16:13:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:13:50.997595 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699b9ede-074f-4510-a065-8346958fab3f None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:50.998408 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699b9ede-074f-4510-a065-8346958fab3f None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:50.998934 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699b9ede-074f-4510-a065-8346958fab3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:51.065393 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-699b9ede-074f-4510-a065-8346958fab3f None None] MFA Rules not processed for user `1f1a985495ed48a4acc48e2e6fc7f977`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:13:51.080496 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-699b9ede-074f-4510-a065-8346958fab3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:51.112217 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbc0ca2-e4eb-476f-ac0f-927e312babb2 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:51.113330 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 872/1747] 10.222.0.57 () {58 vars in 1058 bytes} [Mon Sep 19 16:13:50 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:13:51.122267 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699b9ede-074f-4510-a065-8346958fab3f None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:51.136240 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea5efe7-b344-4e3d-802e-b593eccf4c7e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1a985495ed48a4acc48e2e6fc7f977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:51.138553 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea5efe7-b344-4e3d-802e-b593eccf4c7e service placement] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:51.139269 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea5efe7-b344-4e3d-802e-b593eccf4c7e service placement] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:51.139536 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699b9ede-074f-4510-a065-8346958fab3f None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:51.140204 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea5efe7-b344-4e3d-802e-b593eccf4c7e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:51.148055 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-3ea5efe7-b344-4e3d-802e-b593eccf4c7e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:51.158055 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2efdd5e-1dd3-405f-a941-d368a1e8072e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1a985495ed48a4acc48e2e6fc7f977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:51.158980 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2efdd5e-1dd3-405f-a941-d368a1e8072e service placement] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:51.159402 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2efdd5e-1dd3-405f-a941-d368a1e8072e service placement] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:51.159815 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2efdd5e-1dd3-405f-a941-d368a1e8072e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:51.225235 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2efdd5e-1dd3-405f-a941-d368a1e8072e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:51.228799 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2efdd5e-1dd3-405f-a941-d368a1e8072e service placement] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:51.256508 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea5efe7-b344-4e3d-802e-b593eccf4c7e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:51.262529 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 873/1748] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:13:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 16:13:51.266286 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea5efe7-b344-4e3d-802e-b593eccf4c7e service placement] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:51.308692 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 876/1749] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:13:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:13:52.144619 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635abee3-f50c-40cf-848f-def6a7978402 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:52.145195 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635abee3-f50c-40cf-848f-def6a7978402 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:52.146559 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635abee3-f50c-40cf-848f-def6a7978402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:52.256397 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-635abee3-f50c-40cf-848f-def6a7978402 None None] MFA Rules not processed for user `8cd9c727c0b444cb83497ea4f78e24d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:13:52.282279 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-635abee3-f50c-40cf-848f-def6a7978402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:52.347891 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 874/1750] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:13:52 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:13:53.146682 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea5efe7-b344-4e3d-802e-b593eccf4c7e service placement] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:53.209300 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea5efe7-b344-4e3d-802e-b593eccf4c7e service placement] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:53.255762 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa10d81d-c755-4968-85ba-03e02f5890bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:53.261482 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa10d81d-c755-4968-85ba-03e02f5890bd service cinder] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:53.263198 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa10d81d-c755-4968-85ba-03e02f5890bd service cinder] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:53.263653 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa10d81d-c755-4968-85ba-03e02f5890bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:53.589181 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa10d81d-c755-4968-85ba-03e02f5890bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:53.591149 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa10d81d-c755-4968-85ba-03e02f5890bd service cinder] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:53.658939 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 877/1751] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:13:53 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:13:53.760670 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a7e9f8-0047-435e-9c39-369e6c2a73cb None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:53.761164 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a7e9f8-0047-435e-9c39-369e6c2a73cb None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:53.761236 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a7e9f8-0047-435e-9c39-369e6c2a73cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:53.815327 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-a2a7e9f8-0047-435e-9c39-369e6c2a73cb None None] MFA Rules not processed for user `754c1b5cb47746cf93c52d6dc0b8fa01`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:13:53.822258 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-a2a7e9f8-0047-435e-9c39-369e6c2a73cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:53.902549 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:13:53.985472 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 875/1752] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:13:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:13:56.728932 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa10d81d-c755-4968-85ba-03e02f5890bd service cinder] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:56.957229 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa10d81d-c755-4968-85ba-03e02f5890bd service cinder] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:57.001873 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cfc41d-1d67-4265-8a3b-ff7b9f61b92d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:57.003442 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cfc41d-1d67-4265-8a3b-ff7b9f61b92d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:57.003949 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cfc41d-1d67-4265-8a3b-ff7b9f61b92d admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:57.004450 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cfc41d-1d67-4265-8a3b-ff7b9f61b92d admin admin] PATH_INFO: `/v3/users/739e63f410f94e2f9f59d1705c703fc5` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:57.014017 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cfc41d-1d67-4265-8a3b-ff7b9f61b92d admin admin] RBAC: Authorizing `identity:delete_user(user_id=739e63f410f94e2f9f59d1705c703fc5)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:57.029871 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cfc41d-1d67-4265-8a3b-ff7b9f61b92d admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:57.239538 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:13:57.245240 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-35cfc41d-1d67-4265-8a3b-ff7b9f61b92d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '739e63f410f94e2f9f59d1705c703fc5'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:57.261167 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-35cfc41d-1d67-4265-8a3b-ff7b9f61b92d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '739e63f410f94e2f9f59d1705c703fc5'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:57.296437 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-35cfc41d-1d67-4265-8a3b-ff7b9f61b92d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '739e63f410f94e2f9f59d1705c703fc5'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:57.318249 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-35cfc41d-1d67-4265-8a3b-ff7b9f61b92d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '739e63f410f94e2f9f59d1705c703fc5'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:57.318993 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-35cfc41d-1d67-4265-8a3b-ff7b9f61b92d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '739e63f410f94e2f9f59d1705c703fc5'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:57.334184 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 878/1753] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:13:56 2022] DELETE /identity/v3/users/739e63f410f94e2f9f59d1705c703fc5 => generated 0 bytes in 607 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:13:57.340913 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73885f1d-2a19-4558-97dc-261550c1ffc5 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:57.341753 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73885f1d-2a19-4558-97dc-261550c1ffc5 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:57.342233 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73885f1d-2a19-4558-97dc-261550c1ffc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:57.448182 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-73885f1d-2a19-4558-97dc-261550c1ffc5 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:13:57.460715 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-73885f1d-2a19-4558-97dc-261550c1ffc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:57.654271 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 876/1754] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:13:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:13:57.665467 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cfc41d-1d67-4265-8a3b-ff7b9f61b92d admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:57.667069 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-35cfc41d-1d67-4265-8a3b-ff7b9f61b92d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:57.868297 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cfc41d-1d67-4265-8a3b-ff7b9f61b92d admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:57.878878 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da14131-9a69-4da1-93a2-c07d27f7f597 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:57.879545 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da14131-9a69-4da1-93a2-c07d27f7f597 service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:57.879826 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da14131-9a69-4da1-93a2-c07d27f7f597 service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:57.880123 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da14131-9a69-4da1-93a2-c07d27f7f597 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:57.906787 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da14131-9a69-4da1-93a2-c07d27f7f597 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:57.908092 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da14131-9a69-4da1-93a2-c07d27f7f597 service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:57.930948 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 879/1755] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:13:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:13:58.935621 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a570d-e0bb-4931-b98d-1ac41572c7dd None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:58.935621 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a570d-e0bb-4931-b98d-1ac41572c7dd None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:58.936454 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a570d-e0bb-4931-b98d-1ac41572c7dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:58.971331 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-642a570d-e0bb-4931-b98d-1ac41572c7dd None None] MFA Rules not processed for user `bd190d8e252342ca9232f7a0c966c715`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:13:58.977459 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-642a570d-e0bb-4931-b98d-1ac41572c7dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:58.983430 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da14131-9a69-4da1-93a2-c07d27f7f597 service neutron] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:58.985711 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-3da14131-9a69-4da1-93a2-c07d27f7f597 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:59.000541 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da14131-9a69-4da1-93a2-c07d27f7f597 service neutron] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:59.006563 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c634e0e-4a63-48d2-90ec-d432b013a813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:59.007225 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c634e0e-4a63-48d2-90ec-d432b013a813 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:59.007503 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c634e0e-4a63-48d2-90ec-d432b013a813 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:59.008380 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c634e0e-4a63-48d2-90ec-d432b013a813 admin admin] PATH_INFO: `/v3/projects/2b693f20a7834354a7b4988012a30b99` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:59.009641 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c634e0e-4a63-48d2-90ec-d432b013a813 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b693f20a7834354a7b4988012a30b99)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:59.012614 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c634e0e-4a63-48d2-90ec-d432b013a813 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:59.104329 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-0c634e0e-4a63-48d2-90ec-d432b013a813 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b693f20a7834354a7b4988012a30b99', 'request_id': 'req-0c634e0e-4a63-48d2-90ec-d432b013a813'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:59.135278 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-0c634e0e-4a63-48d2-90ec-d432b013a813 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b693f20a7834354a7b4988012a30b99', 'request_id': 'req-0c634e0e-4a63-48d2-90ec-d432b013a813'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:59.154202 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-0c634e0e-4a63-48d2-90ec-d432b013a813 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b693f20a7834354a7b4988012a30b99', 'request_id': 'req-0c634e0e-4a63-48d2-90ec-d432b013a813'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:59.184168 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-0c634e0e-4a63-48d2-90ec-d432b013a813 admin admin] The token cache is being invalidate because project 2b693f20a7834354a7b4988012a30b99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81618) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:13:59.184996 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-0c634e0e-4a63-48d2-90ec-d432b013a813 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:59.186118 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-0c634e0e-4a63-48d2-90ec-d432b013a813 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:59.189141 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 880/1756] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:13:58 2022] DELETE /identity/v3/projects/2b693f20a7834354a7b4988012a30b99 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 16:13:59.195013 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c634e0e-4a63-48d2-90ec-d432b013a813 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:13:59.198081 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-0c634e0e-4a63-48d2-90ec-d432b013a813 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:13:59.205677 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 877/1757] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:13:58 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:13:59.328650 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:13:59.354222 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c634e0e-4a63-48d2-90ec-d432b013a813 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:13:59.366146 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cc8c64-c7d1-41ba-aa03-fa14d106abed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:13:59.367534 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc8c64-c7d1-41ba-aa03-fa14d106abed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:13:59.368186 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc8c64-c7d1-41ba-aa03-fa14d106abed admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:13:59.381174 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc8c64-c7d1-41ba-aa03-fa14d106abed admin admin] PATH_INFO: `/v3/users/75878aa281c64627959206b564c7ead8` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:13:59.383873 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cc8c64-c7d1-41ba-aa03-fa14d106abed admin admin] RBAC: Authorizing `identity:delete_user(user_id=75878aa281c64627959206b564c7ead8)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:13:59.388607 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cc8c64-c7d1-41ba-aa03-fa14d106abed admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:13:59.480123 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-a3cc8c64-c7d1-41ba-aa03-fa14d106abed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75878aa281c64627959206b564c7ead8'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:59.495732 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-a3cc8c64-c7d1-41ba-aa03-fa14d106abed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75878aa281c64627959206b564c7ead8'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:59.515144 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-a3cc8c64-c7d1-41ba-aa03-fa14d106abed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75878aa281c64627959206b564c7ead8'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:59.527171 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-a3cc8c64-c7d1-41ba-aa03-fa14d106abed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75878aa281c64627959206b564c7ead8'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:59.528381 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-a3cc8c64-c7d1-41ba-aa03-fa14d106abed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75878aa281c64627959206b564c7ead8'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:13:59.537750 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 881/1758] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:13:59 2022] DELETE /identity/v3/users/75878aa281c64627959206b564c7ead8 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 16:14:00.426140 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642a570d-e0bb-4931-b98d-1ac41572c7dd None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:00.465665 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642a570d-e0bb-4931-b98d-1ac41572c7dd None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:00.591824 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e7edd2-e18b-4aad-b07b-46630ad4655e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:00.592925 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e7edd2-e18b-4aad-b07b-46630ad4655e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:00.593403 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e7edd2-e18b-4aad-b07b-46630ad4655e admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:00.593898 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e7edd2-e18b-4aad-b07b-46630ad4655e admin admin] PATH_INFO: `/v3/projects/d12f082dcbd34ea781bd046fa64664ec` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:00.596295 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e7edd2-e18b-4aad-b07b-46630ad4655e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d12f082dcbd34ea781bd046fa64664ec)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:00.603045 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e7edd2-e18b-4aad-b07b-46630ad4655e admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:00.739481 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:00.843727 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-53e7edd2-e18b-4aad-b07b-46630ad4655e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd12f082dcbd34ea781bd046fa64664ec', 'request_id': 'req-53e7edd2-e18b-4aad-b07b-46630ad4655e'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:00.861760 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-53e7edd2-e18b-4aad-b07b-46630ad4655e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd12f082dcbd34ea781bd046fa64664ec', 'request_id': 'req-53e7edd2-e18b-4aad-b07b-46630ad4655e'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:00.885202 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-53e7edd2-e18b-4aad-b07b-46630ad4655e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd12f082dcbd34ea781bd046fa64664ec', 'request_id': 'req-53e7edd2-e18b-4aad-b07b-46630ad4655e'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:00.910639 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-53e7edd2-e18b-4aad-b07b-46630ad4655e admin admin] The token cache is being invalidate because project d12f082dcbd34ea781bd046fa64664ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81617) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:14:00.911272 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-53e7edd2-e18b-4aad-b07b-46630ad4655e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:00.912827 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-53e7edd2-e18b-4aad-b07b-46630ad4655e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:00.918987 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 878/1759] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:14:00 2022] DELETE /identity/v3/projects/d12f082dcbd34ea781bd046fa64664ec => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:14:00.924785 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412a29fd-1ac6-4bda-9223-ad0bc298768f None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:00.925405 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412a29fd-1ac6-4bda-9223-ad0bc298768f None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:00.925625 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412a29fd-1ac6-4bda-9223-ad0bc298768f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:00.979776 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-412a29fd-1ac6-4bda-9223-ad0bc298768f None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:14:00.981328 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-412a29fd-1ac6-4bda-9223-ad0bc298768f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:01.126104 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 882/1760] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:14:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:14:01.140292 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f2a88-1c6c-4973-aa23-b4b1427f2b9c None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:01.140799 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f2a88-1c6c-4973-aa23-b4b1427f2b9c None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:01.141308 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f2a88-1c6c-4973-aa23-b4b1427f2b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:01.190144 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-c10f2a88-1c6c-4973-aa23-b4b1427f2b9c None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:14:01.191774 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-c10f2a88-1c6c-4973-aa23-b4b1427f2b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:01.205419 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 879/1761] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:14:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:14:01.206778 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412a29fd-1ac6-4bda-9223-ad0bc298768f None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:01.224132 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412a29fd-1ac6-4bda-9223-ad0bc298768f None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:01.257404 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a6a59f-4d46-4974-97c6-0ed831251bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:01.258979 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a6a59f-4d46-4974-97c6-0ed831251bab admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:01.260262 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a6a59f-4d46-4974-97c6-0ed831251bab admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:01.262000 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a6a59f-4d46-4974-97c6-0ed831251bab admin admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:01.263926 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a6a59f-4d46-4974-97c6-0ed831251bab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:14:01.266076 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a6a59f-4d46-4974-97c6-0ed831251bab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:01.272346 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a6a59f-4d46-4974-97c6-0ed831251bab admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:01.311387 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 883/1762] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:14:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:14:01.316427 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10f2a88-1c6c-4973-aa23-b4b1427f2b9c None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:01.343650 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10f2a88-1c6c-4973-aa23-b4b1427f2b9c None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:01.353275 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea977872-b9a8-4778-8685-2c9ef76246eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:01.359168 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea977872-b9a8-4778-8685-2c9ef76246eb admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:01.359783 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea977872-b9a8-4778-8685-2c9ef76246eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:01.361462 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea977872-b9a8-4778-8685-2c9ef76246eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:01.362270 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea977872-b9a8-4778-8685-2c9ef76246eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:01.368093 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea977872-b9a8-4778-8685-2c9ef76246eb admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:01.448800 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 880/1763] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:14:01 2022] POST /identity/v3/projects => generated 399 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:14:01.456482 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a6a59f-4d46-4974-97c6-0ed831251bab admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:01.485504 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a6a59f-4d46-4974-97c6-0ed831251bab admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:01.708547 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:01.740647 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e14e573-44ef-4627-a9bc-d0cea76c7aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:01.741382 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14e573-44ef-4627-a9bc-d0cea76c7aa3 admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:01.741764 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14e573-44ef-4627-a9bc-d0cea76c7aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:01.742354 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14e573-44ef-4627-a9bc-d0cea76c7aa3 admin admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:01.743209 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e14e573-44ef-4627-a9bc-d0cea76c7aa3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:01.745409 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e14e573-44ef-4627-a9bc-d0cea76c7aa3 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:01.867071 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 884/1764] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:14:01 2022] POST /identity/v3/users => generated 300 bytes in 414 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:14:01.873932 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea977872-b9a8-4778-8685-2c9ef76246eb admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:01.930699 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea977872-b9a8-4778-8685-2c9ef76246eb admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:01.977691 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eec6a8-57b5-4692-bbe1-ca453da4ed5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:01.979892 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eec6a8-57b5-4692-bbe1-ca453da4ed5e admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:01.982733 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eec6a8-57b5-4692-bbe1-ca453da4ed5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:01.987175 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eec6a8-57b5-4692-bbe1-ca453da4ed5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:01.993494 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eec6a8-57b5-4692-bbe1-ca453da4ed5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:02.017847 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eec6a8-57b5-4692-bbe1-ca453da4ed5e admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:02.021294 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d97dc-9692-4d62-98ab-9b1eb5a9b48e None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:02.025497 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d97dc-9692-4d62-98ab-9b1eb5a9b48e None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:02.026577 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d97dc-9692-4d62-98ab-9b1eb5a9b48e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:02.101212 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 881/1765] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:14:01 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 16:14:02.104074 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eec6a8-57b5-4692-bbe1-ca453da4ed5e admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:02.133836 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eec6a8-57b5-4692-bbe1-ca453da4ed5e admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:02.149987 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f9b350-fa23-48c2-b9fd-bb2d4ecbbb85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:02.182404 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f9b350-fa23-48c2-b9fd-bb2d4ecbbb85 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:02.182762 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f9b350-fa23-48c2-b9fd-bb2d4ecbbb85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:02.184419 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f9b350-fa23-48c2-b9fd-bb2d4ecbbb85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:02.186017 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f9b350-fa23-48c2-b9fd-bb2d4ecbbb85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:02.199026 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f9b350-fa23-48c2-b9fd-bb2d4ecbbb85 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:02.242835 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 882/1766] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:14:02 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:14:02.249348 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f9b350-fa23-48c2-b9fd-bb2d4ecbbb85 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:02.292878 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f9b350-fa23-48c2-b9fd-bb2d4ecbbb85 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:02.311609 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f48bb4-0029-434a-835c-472db39fe906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:02.323749 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f48bb4-0029-434a-835c-472db39fe906 admin admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:02.327160 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f48bb4-0029-434a-835c-472db39fe906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:02.328465 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f48bb4-0029-434a-835c-472db39fe906 admin admin] PATH_INFO: `/v3/projects/359462b10d7b4096b98f543da425b5aa/users/0387a24e9b7a420aa25053eff731edc6/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:02.336538 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-897d97dc-9692-4d62-98ab-9b1eb5a9b48e None None] MFA Rules not processed for user `7222bd3b313b47e5a4fa0ebc9ff0cdd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:14:02.341991 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-897d97dc-9692-4d62-98ab-9b1eb5a9b48e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:02.416628 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f48bb4-0029-434a-835c-472db39fe906 admin admin] RBAC: Authorizing `identity:create_grant(project_id=359462b10d7b4096b98f543da425b5aa, user_id=0387a24e9b7a420aa25053eff731edc6, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:02.428127 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f48bb4-0029-434a-835c-472db39fe906 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:02.544981 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 883/1767] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:14:02 2022] PUT /identity/v3/projects/359462b10d7b4096b98f543da425b5aa/users/0387a24e9b7a420aa25053eff731edc6/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:14:02.558010 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2bf8b7-25fc-46a7-a7b4-4660a96f2241 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:02.558659 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2bf8b7-25fc-46a7-a7b4-4660a96f2241 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:02.559112 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2bf8b7-25fc-46a7-a7b4-4660a96f2241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:02.576258 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 885/1768] 10.222.0.57 () {58 vars in 1009 bytes} [Mon Sep 19 16:14:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 557 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:14:02.677682 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-dc2bf8b7-25fc-46a7-a7b4-4660a96f2241 None None] MFA Rules not processed for user `0387a24e9b7a420aa25053eff731edc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:14:02.681366 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-dc2bf8b7-25fc-46a7-a7b4-4660a96f2241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:02.822819 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897d97dc-9692-4d62-98ab-9b1eb5a9b48e None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:02.825595 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-897d97dc-9692-4d62-98ab-9b1eb5a9b48e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:02.870014 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 884/1769] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:14:02 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:14:02.871104 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2bf8b7-25fc-46a7-a7b4-4660a96f2241 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:03.009978 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:03.086941 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897d97dc-9692-4d62-98ab-9b1eb5a9b48e None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:03.180119 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2bf8b7-25fc-46a7-a7b4-4660a96f2241 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:03.201493 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5d3e83-1eda-4115-8bbb-472a020ae285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:03.202310 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d3e83-1eda-4115-8bbb-472a020ae285 admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:03.202690 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d3e83-1eda-4115-8bbb-472a020ae285 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:03.203315 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d3e83-1eda-4115-8bbb-472a020ae285 admin admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:03.203987 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5d3e83-1eda-4115-8bbb-472a020ae285 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:03.208194 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:03.212803 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5d3e83-1eda-4115-8bbb-472a020ae285 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:03.287526 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a13fae-688b-4fad-b3df-6d03aa2ca979 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '884a0524aca744b99c7496c11c3291ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:03.289558 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a13fae-688b-4fad-b3df-6d03aa2ca979 service swift] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:03.290121 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a13fae-688b-4fad-b3df-6d03aa2ca979 service swift] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:03.290710 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a13fae-688b-4fad-b3df-6d03aa2ca979 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:03.295873 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 885/1770] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:14:02 2022] POST /identity/v3/projects => generated 399 bytes in 430 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:14:03.304067 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5d3e83-1eda-4115-8bbb-472a020ae285 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:03.455382 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a13fae-688b-4fad-b3df-6d03aa2ca979 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:03.461896 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a13fae-688b-4fad-b3df-6d03aa2ca979 service swift] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:03.501290 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 886/1771] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Sep 19 16:14:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 681 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 16:14:03.652105 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5d3e83-1eda-4115-8bbb-472a020ae285 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:03.665680 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae80f82-ab21-4443-a73b-73f50e76925c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:03.666539 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae80f82-ab21-4443-a73b-73f50e76925c admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:03.666966 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae80f82-ab21-4443-a73b-73f50e76925c admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:03.667723 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae80f82-ab21-4443-a73b-73f50e76925c admin admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:03.668587 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae80f82-ab21-4443-a73b-73f50e76925c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:03.678554 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae80f82-ab21-4443-a73b-73f50e76925c admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:03.739849 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 886/1772] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:14:03 2022] POST /identity/v3/users => generated 306 bytes in 440 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:14:03.744950 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a13fae-688b-4fad-b3df-6d03aa2ca979 service swift] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:03.778470 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a13fae-688b-4fad-b3df-6d03aa2ca979 service swift] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:03.814904 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b440995-2e47-4db8-8931-6ca5d21a3bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:03.816004 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b440995-2e47-4db8-8931-6ca5d21a3bef admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:03.816493 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b440995-2e47-4db8-8931-6ca5d21a3bef admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:03.817065 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b440995-2e47-4db8-8931-6ca5d21a3bef admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:03.818184 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b440995-2e47-4db8-8931-6ca5d21a3bef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:03.821823 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b440995-2e47-4db8-8931-6ca5d21a3bef admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:03.869305 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 887/1773] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:14:03 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:14:03.884371 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae80f82-ab21-4443-a73b-73f50e76925c admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:03.906685 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae80f82-ab21-4443-a73b-73f50e76925c admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:03.945670 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc82d91-3e99-416c-86f1-360626d74fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:03.951754 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc82d91-3e99-416c-86f1-360626d74fbf admin admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:03.951754 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc82d91-3e99-416c-86f1-360626d74fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:03.958559 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc82d91-3e99-416c-86f1-360626d74fbf admin admin] PATH_INFO: `/v3/projects/d2aceea319824fffba0aca6673bfc396/users/43ed7b1ee54642d49912527d8bcc9a05/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:04.062611 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc82d91-3e99-416c-86f1-360626d74fbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2aceea319824fffba0aca6673bfc396, user_id=43ed7b1ee54642d49912527d8bcc9a05, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:04.079541 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc82d91-3e99-416c-86f1-360626d74fbf admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:04.136690 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 887/1774] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:14:03 2022] PUT /identity/v3/projects/d2aceea319824fffba0aca6673bfc396/users/43ed7b1ee54642d49912527d8bcc9a05/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:14:04.141463 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b440995-2e47-4db8-8931-6ca5d21a3bef admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:04.194992 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b440995-2e47-4db8-8931-6ca5d21a3bef admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:04.561629 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e8fc96-f406-4126-a484-e4dbed9cfafe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:04.562721 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e8fc96-f406-4126-a484-e4dbed9cfafe admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:04.563199 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e8fc96-f406-4126-a484-e4dbed9cfafe admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:04.563690 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e8fc96-f406-4126-a484-e4dbed9cfafe admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:04.564891 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e8fc96-f406-4126-a484-e4dbed9cfafe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:04.568027 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e8fc96-f406-4126-a484-e4dbed9cfafe admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:04.623976 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc82d91-3e99-416c-86f1-360626d74fbf admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:04.626968 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 888/1775] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:14:04 2022] GET /identity/v3/roles => generated 1420 bytes in 487 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:14:04.649665 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc82d91-3e99-416c-86f1-360626d74fbf admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:04.674605 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04340c5-3241-4f5e-8aef-03efbeeead83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:04.676487 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04340c5-3241-4f5e-8aef-03efbeeead83 admin admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:04.678551 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04340c5-3241-4f5e-8aef-03efbeeead83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:04.679066 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04340c5-3241-4f5e-8aef-03efbeeead83 admin admin] PATH_INFO: `/v3/projects/d2aceea319824fffba0aca6673bfc396/users/43ed7b1ee54642d49912527d8bcc9a05/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:04.684434 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04340c5-3241-4f5e-8aef-03efbeeead83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2aceea319824fffba0aca6673bfc396, user_id=43ed7b1ee54642d49912527d8bcc9a05, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:04.690412 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04340c5-3241-4f5e-8aef-03efbeeead83 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:04.722003 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 888/1776] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:14:04 2022] PUT /identity/v3/projects/d2aceea319824fffba0aca6673bfc396/users/43ed7b1ee54642d49912527d8bcc9a05/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:14:04.734184 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db5749-b121-4308-91dc-0d1fb93d4714 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:04.734715 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db5749-b121-4308-91dc-0d1fb93d4714 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:04.735198 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db5749-b121-4308-91dc-0d1fb93d4714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:04.843889 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-52db5749-b121-4308-91dc-0d1fb93d4714 None None] MFA Rules not processed for user `43ed7b1ee54642d49912527d8bcc9a05`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:14:04.845512 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-52db5749-b121-4308-91dc-0d1fb93d4714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:05.042263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:05.086113 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 889/1777] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:14:04 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:14:05.102238 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04340c5-3241-4f5e-8aef-03efbeeead83 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:05.105454 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-e04340c5-3241-4f5e-8aef-03efbeeead83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:05.424401 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04340c5-3241-4f5e-8aef-03efbeeead83 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:05.470821 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bfdad5-fcd5-4810-a39b-48509edd4e83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:05.471470 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bfdad5-fcd5-4810-a39b-48509edd4e83 service cinder] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:05.471752 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bfdad5-fcd5-4810-a39b-48509edd4e83 service cinder] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:05.472031 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bfdad5-fcd5-4810-a39b-48509edd4e83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:05.529237 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bfdad5-fcd5-4810-a39b-48509edd4e83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:05.530865 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bfdad5-fcd5-4810-a39b-48509edd4e83 service cinder] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:05.551847 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 889/1778] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:14:05 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:14:05.765044 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52db5749-b121-4308-91dc-0d1fb93d4714 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:05.780795 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52db5749-b121-4308-91dc-0d1fb93d4714 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:05.785973 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b8d94a-546e-45b8-bf40-cbfaf26baf7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:05.785973 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b8d94a-546e-45b8-bf40-cbfaf26baf7f service cinder] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:05.785973 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b8d94a-546e-45b8-bf40-cbfaf26baf7f service cinder] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:05.786461 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b8d94a-546e-45b8-bf40-cbfaf26baf7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:05.876859 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b8d94a-546e-45b8-bf40-cbfaf26baf7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:05.879105 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b8d94a-546e-45b8-bf40-cbfaf26baf7f service cinder] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:05.893888 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 890/1779] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:14:05 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:14:08.033614 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4810763-9f64-46c1-a578-258ef03c2b4a None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:08.034766 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4810763-9f64-46c1-a578-258ef03c2b4a None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:08.035733 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4810763-9f64-46c1-a578-258ef03c2b4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:08.221356 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-c4810763-9f64-46c1-a578-258ef03c2b4a None None] MFA Rules not processed for user `50355d7b1c6c4f0bbbbc7fe56efb9039`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:14:08.239616 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-c4810763-9f64-46c1-a578-258ef03c2b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:08.256304 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:08.592308 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 890/1780] 10.222.0.16 () {58 vars in 1015 bytes} [Mon Sep 19 16:14:08 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 560 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:14:08.670633 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b8d94a-546e-45b8-bf40-cbfaf26baf7f service cinder] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:08.674136 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-24b8d94a-546e-45b8-bf40-cbfaf26baf7f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:08.711651 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4810763-9f64-46c1-a578-258ef03c2b4a None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:08.714009 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-c4810763-9f64-46c1-a578-258ef03c2b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:08.786615 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b8d94a-546e-45b8-bf40-cbfaf26baf7f service cinder] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:08.794798 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e1dc38-0e6c-4cf0-bd44-9b5b039380aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:08.796565 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e1dc38-0e6c-4cf0-bd44-9b5b039380aa service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:08.799742 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e1dc38-0e6c-4cf0-bd44-9b5b039380aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:08.802663 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e1dc38-0e6c-4cf0-bd44-9b5b039380aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:08.813699 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4810763-9f64-46c1-a578-258ef03c2b4a None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:08.852906 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd292ef-d0d5-4444-a824-cde110ab1dc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:08.858195 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd292ef-d0d5-4444-a824-cde110ab1dc3 service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:08.858195 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd292ef-d0d5-4444-a824-cde110ab1dc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:08.858195 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd292ef-d0d5-4444-a824-cde110ab1dc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:08.891979 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e1dc38-0e6c-4cf0-bd44-9b5b039380aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:08.895326 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e1dc38-0e6c-4cf0-bd44-9b5b039380aa service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:08.921507 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd292ef-d0d5-4444-a824-cde110ab1dc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:08.926034 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd292ef-d0d5-4444-a824-cde110ab1dc3 service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:08.971516 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 891/1781] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:14:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:14:08.981466 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd292ef-d0d5-4444-a824-cde110ab1dc3 service neutron] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:09.002103 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 891/1782] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:14:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:14:09.069959 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd292ef-d0d5-4444-a824-cde110ab1dc3 service neutron] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:09.106776 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b44ea8-8340-4363-b1c5-22de166b0357 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:09.113141 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b44ea8-8340-4363-b1c5-22de166b0357 service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:09.123247 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b44ea8-8340-4363-b1c5-22de166b0357 service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:09.123595 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b44ea8-8340-4363-b1c5-22de166b0357 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:09.174591 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b44ea8-8340-4363-b1c5-22de166b0357 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:09.176592 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b44ea8-8340-4363-b1c5-22de166b0357 service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:09.232715 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 892/1783] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:14:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:14:09.527855 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e1dc38-0e6c-4cf0-bd44-9b5b039380aa service neutron] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:09.529424 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-83e1dc38-0e6c-4cf0-bd44-9b5b039380aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:09.910569 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e1dc38-0e6c-4cf0-bd44-9b5b039380aa service neutron] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:09.918038 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c431bcc-d60c-44a9-9a3f-ae27c45cda62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:09.919357 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c431bcc-d60c-44a9-9a3f-ae27c45cda62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:09.919897 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c431bcc-d60c-44a9-9a3f-ae27c45cda62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:09.920594 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c431bcc-d60c-44a9-9a3f-ae27c45cda62 admin admin] PATH_INFO: `/v3/users/e28246a7df3d4fbd9e7aa98fb293d807` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:09.922213 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c431bcc-d60c-44a9-9a3f-ae27c45cda62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e28246a7df3d4fbd9e7aa98fb293d807)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:09.924931 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c431bcc-d60c-44a9-9a3f-ae27c45cda62 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:10.056392 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-0c431bcc-d60c-44a9-9a3f-ae27c45cda62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e28246a7df3d4fbd9e7aa98fb293d807'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:10.084764 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-0c431bcc-d60c-44a9-9a3f-ae27c45cda62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e28246a7df3d4fbd9e7aa98fb293d807'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:10.141392 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-0c431bcc-d60c-44a9-9a3f-ae27c45cda62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e28246a7df3d4fbd9e7aa98fb293d807'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:10.163035 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:10.175797 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-0c431bcc-d60c-44a9-9a3f-ae27c45cda62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e28246a7df3d4fbd9e7aa98fb293d807'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:10.176541 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-0c431bcc-d60c-44a9-9a3f-ae27c45cda62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e28246a7df3d4fbd9e7aa98fb293d807'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:10.208143 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 892/1784] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:14:09 2022] DELETE /identity/v3/users/e28246a7df3d4fbd9e7aa98fb293d807 => generated 0 bytes in 681 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:14:10.213518 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113261f9-3ce5-4204-b3e3-715ac4d7c555 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:10.214862 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113261f9-3ce5-4204-b3e3-715ac4d7c555 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:10.214862 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113261f9-3ce5-4204-b3e3-715ac4d7c555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:10.268123 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-113261f9-3ce5-4204-b3e3-715ac4d7c555 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:14:10.271961 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-113261f9-3ce5-4204-b3e3-715ac4d7c555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:10.553405 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 893/1785] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:14:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:14:10.625917 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c431bcc-d60c-44a9-9a3f-ae27c45cda62 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:10.628018 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-0c431bcc-d60c-44a9-9a3f-ae27c45cda62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:10.917842 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c431bcc-d60c-44a9-9a3f-ae27c45cda62 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:10.962852 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a1709b-487c-4204-972d-0abaca88b68d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:10.963665 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1709b-487c-4204-972d-0abaca88b68d service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:10.964126 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1709b-487c-4204-972d-0abaca88b68d service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:10.964597 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1709b-487c-4204-972d-0abaca88b68d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:11.035744 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a1709b-487c-4204-972d-0abaca88b68d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:11.038411 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a1709b-487c-4204-972d-0abaca88b68d service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:11.111505 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 893/1786] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:14:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:14:12.259935 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113261f9-3ce5-4204-b3e3-715ac4d7c555 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:12.263859 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-113261f9-3ce5-4204-b3e3-715ac4d7c555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:12.294347 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113261f9-3ce5-4204-b3e3-715ac4d7c555 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:12.302290 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b4d3ca-ab78-41b0-9a34-44c3c3d8dc6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:12.303674 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b4d3ca-ab78-41b0-9a34-44c3c3d8dc6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:12.304226 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b4d3ca-ab78-41b0-9a34-44c3c3d8dc6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:12.305078 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b4d3ca-ab78-41b0-9a34-44c3c3d8dc6a admin admin] PATH_INFO: `/v3/projects/3fca28b39fc84e509f256920c800d468` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:12.307786 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b4d3ca-ab78-41b0-9a34-44c3c3d8dc6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fca28b39fc84e509f256920c800d468)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:12.310809 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b4d3ca-ab78-41b0-9a34-44c3c3d8dc6a admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:12.466790 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-a8b4d3ca-ab78-41b0-9a34-44c3c3d8dc6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fca28b39fc84e509f256920c800d468', 'request_id': 'req-a8b4d3ca-ab78-41b0-9a34-44c3c3d8dc6a'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:12.483915 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-a8b4d3ca-ab78-41b0-9a34-44c3c3d8dc6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fca28b39fc84e509f256920c800d468', 'request_id': 'req-a8b4d3ca-ab78-41b0-9a34-44c3c3d8dc6a'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:12.501825 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-a8b4d3ca-ab78-41b0-9a34-44c3c3d8dc6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fca28b39fc84e509f256920c800d468', 'request_id': 'req-a8b4d3ca-ab78-41b0-9a34-44c3c3d8dc6a'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:12.539322 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-a8b4d3ca-ab78-41b0-9a34-44c3c3d8dc6a admin admin] The token cache is being invalidate because project 3fca28b39fc84e509f256920c800d468 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81617) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:14:12.539860 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-a8b4d3ca-ab78-41b0-9a34-44c3c3d8dc6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:12.541540 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-a8b4d3ca-ab78-41b0-9a34-44c3c3d8dc6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:12.561623 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 894/1787] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:14:12 2022] DELETE /identity/v3/projects/3fca28b39fc84e509f256920c800d468 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:14:12.565697 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a1709b-487c-4204-972d-0abaca88b68d service neutron] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:12.598180 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a1709b-487c-4204-972d-0abaca88b68d service neutron] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:12.689563 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01ee301-7c2a-47a0-a85b-85b9c765a9c3 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:12.690063 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01ee301-7c2a-47a0-a85b-85b9c765a9c3 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:12.690591 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01ee301-7c2a-47a0-a85b-85b9c765a9c3 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:12.695434 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 895/1788] 10.222.0.57 () {54 vars in 921 bytes} [Mon Sep 19 16:14:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:14:12.738820 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7d7068-973a-443b-8045-ccb2db926aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:12.739505 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7d7068-973a-443b-8045-ccb2db926aa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:12.739793 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7d7068-973a-443b-8045-ccb2db926aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:12.740069 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7d7068-973a-443b-8045-ccb2db926aa5 admin admin] PATH_INFO: `/v3/users/1b27b454d2d944e29aee8329564296c7` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:12.741306 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7d7068-973a-443b-8045-ccb2db926aa5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b27b454d2d944e29aee8329564296c7)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:12.743901 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7d7068-973a-443b-8045-ccb2db926aa5 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:12.791319 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a85cc39-55d7-49cf-85e5-353adf327add None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:12.791319 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a85cc39-55d7-49cf-85e5-353adf327add None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:12.791319 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a85cc39-55d7-49cf-85e5-353adf327add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:12.828352 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-0a85cc39-55d7-49cf-85e5-353adf327add None None] MFA Rules not processed for user `1ecd01cc72834b72b5ff2c4b47249737`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:14:12.830352 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-0a85cc39-55d7-49cf-85e5-353adf327add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:12.831695 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-8a7d7068-973a-443b-8045-ccb2db926aa5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b27b454d2d944e29aee8329564296c7'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:12.848034 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-8a7d7068-973a-443b-8045-ccb2db926aa5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b27b454d2d944e29aee8329564296c7'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:12.864174 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-8a7d7068-973a-443b-8045-ccb2db926aa5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b27b454d2d944e29aee8329564296c7'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:12.877611 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-8a7d7068-973a-443b-8045-ccb2db926aa5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b27b454d2d944e29aee8329564296c7'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:12.883117 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-8a7d7068-973a-443b-8045-ccb2db926aa5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b27b454d2d944e29aee8329564296c7'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:12.887955 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:12.918313 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 894/1789] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:14:12 2022] DELETE /identity/v3/users/1b27b454d2d944e29aee8329564296c7 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:14:12.984749 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 896/1790] 10.222.0.57 () {58 vars in 1018 bytes} [Mon Sep 19 16:14:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:14:13.401919 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7d7068-973a-443b-8045-ccb2db926aa5 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:13.403973 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-8a7d7068-973a-443b-8045-ccb2db926aa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:13.433272 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:13.570922 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7d7068-973a-443b-8045-ccb2db926aa5 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:13.582490 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52213a5-e365-4adc-92d0-25adb3b09113 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:13.584920 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52213a5-e365-4adc-92d0-25adb3b09113 service nova] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:13.588236 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52213a5-e365-4adc-92d0-25adb3b09113 service nova] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:13.589029 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52213a5-e365-4adc-92d0-25adb3b09113 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:13.623191 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52213a5-e365-4adc-92d0-25adb3b09113 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:13.625226 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52213a5-e365-4adc-92d0-25adb3b09113 service nova] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:13.682489 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 895/1791] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:14:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:14:13.987067 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceecd1d7-b80c-4fed-a57c-82b72a531386 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:13.993140 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceecd1d7-b80c-4fed-a57c-82b72a531386 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:13.996280 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceecd1d7-b80c-4fed-a57c-82b72a531386 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:14.011319 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 897/1792] 10.222.0.57 () {54 vars in 921 bytes} [Mon Sep 19 16:14:13 2022] GET /identity => generated 270 bytes in 36 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:14:14.029376 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15ad344-da0d-4da4-866a-2a8df0f1a2eb None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:14.030749 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15ad344-da0d-4da4-866a-2a8df0f1a2eb None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:14.031260 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15ad344-da0d-4da4-866a-2a8df0f1a2eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:14.083718 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-f15ad344-da0d-4da4-866a-2a8df0f1a2eb None None] MFA Rules not processed for user `1ecd01cc72834b72b5ff2c4b47249737`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:14:14.085291 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-f15ad344-da0d-4da4-866a-2a8df0f1a2eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:14.107848 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 896/1793] 10.222.0.57 () {58 vars in 1018 bytes} [Mon Sep 19 16:14:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:14:14.276666 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceecd1d7-b80c-4fed-a57c-82b72a531386 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:14.279385 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-ceecd1d7-b80c-4fed-a57c-82b72a531386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:14.341277 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceecd1d7-b80c-4fed-a57c-82b72a531386 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:14.354534 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3437c2-acd6-4480-a067-5a6729eb4c27 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:14.356002 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3437c2-acd6-4480-a067-5a6729eb4c27 service nova] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:14.358006 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3437c2-acd6-4480-a067-5a6729eb4c27 service nova] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:14.359038 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3437c2-acd6-4480-a067-5a6729eb4c27 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:14.383945 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3437c2-acd6-4480-a067-5a6729eb4c27 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:14.390776 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3437c2-acd6-4480-a067-5a6729eb4c27 service nova] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:14.446651 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 898/1794] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:14:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:14:14.761683 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15ad344-da0d-4da4-866a-2a8df0f1a2eb None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:14.765227 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-f15ad344-da0d-4da4-866a-2a8df0f1a2eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:15.130909 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15ad344-da0d-4da4-866a-2a8df0f1a2eb None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:15.151663 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778d065a-595d-484b-b9d0-d2d185b85542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:15.152322 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d065a-595d-484b-b9d0-d2d185b85542 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:15.152627 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d065a-595d-484b-b9d0-d2d185b85542 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:15.152902 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d065a-595d-484b-b9d0-d2d185b85542 admin admin] PATH_INFO: `/v3/users/0387a24e9b7a420aa25053eff731edc6` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:15.154166 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778d065a-595d-484b-b9d0-d2d185b85542 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0387a24e9b7a420aa25053eff731edc6)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:15.156272 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778d065a-595d-484b-b9d0-d2d185b85542 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:15.262982 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-778d065a-595d-484b-b9d0-d2d185b85542 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0387a24e9b7a420aa25053eff731edc6'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:15.285682 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-778d065a-595d-484b-b9d0-d2d185b85542 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0387a24e9b7a420aa25053eff731edc6'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:15.312650 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-778d065a-595d-484b-b9d0-d2d185b85542 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0387a24e9b7a420aa25053eff731edc6'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:15.329098 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-778d065a-595d-484b-b9d0-d2d185b85542 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0387a24e9b7a420aa25053eff731edc6'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:15.329650 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-778d065a-595d-484b-b9d0-d2d185b85542 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0387a24e9b7a420aa25053eff731edc6'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:15.344033 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 897/1795] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:14:14 2022] DELETE /identity/v3/users/0387a24e9b7a420aa25053eff731edc6 => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:14:15.350439 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db5bc52-b7c8-4c38-beae-0c6c8dddcb05 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:15.351730 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db5bc52-b7c8-4c38-beae-0c6c8dddcb05 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:15.351730 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db5bc52-b7c8-4c38-beae-0c6c8dddcb05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:15.389936 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-0db5bc52-b7c8-4c38-beae-0c6c8dddcb05 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:14:15.391522 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-0db5bc52-b7c8-4c38-beae-0c6c8dddcb05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:15.623998 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 899/1796] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:14:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:14:15.691420 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778d065a-595d-484b-b9d0-d2d185b85542 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:15.693297 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-778d065a-595d-484b-b9d0-d2d185b85542 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:15.753349 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:15.872297 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778d065a-595d-484b-b9d0-d2d185b85542 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:15.883022 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ca4d96-578f-42ca-8c9a-dab74cc60297 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:15.883671 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ca4d96-578f-42ca-8c9a-dab74cc60297 service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:15.885147 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ca4d96-578f-42ca-8c9a-dab74cc60297 service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:15.885147 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ca4d96-578f-42ca-8c9a-dab74cc60297 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:15.906873 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ca4d96-578f-42ca-8c9a-dab74cc60297 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:15.908377 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ca4d96-578f-42ca-8c9a-dab74cc60297 service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:15.946903 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 898/1797] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:14:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:14:16.482075 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db5bc52-b7c8-4c38-beae-0c6c8dddcb05 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:16.495256 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-0db5bc52-b7c8-4c38-beae-0c6c8dddcb05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:16.623532 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db5bc52-b7c8-4c38-beae-0c6c8dddcb05 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:16.667794 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d3c56d-cb1c-4690-ae6b-69b090e7559a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:16.669667 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d3c56d-cb1c-4690-ae6b-69b090e7559a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:16.673955 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d3c56d-cb1c-4690-ae6b-69b090e7559a admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:16.674482 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d3c56d-cb1c-4690-ae6b-69b090e7559a admin admin] PATH_INFO: `/v3/projects/dc9d27fbfcc347cc9960a4802fb7e8b8` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:16.677179 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d3c56d-cb1c-4690-ae6b-69b090e7559a admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc9d27fbfcc347cc9960a4802fb7e8b8)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:16.683049 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d3c56d-cb1c-4690-ae6b-69b090e7559a admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:16.864829 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-69d3c56d-cb1c-4690-ae6b-69b090e7559a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc9d27fbfcc347cc9960a4802fb7e8b8', 'request_id': 'req-69d3c56d-cb1c-4690-ae6b-69b090e7559a'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:16.875687 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-69d3c56d-cb1c-4690-ae6b-69b090e7559a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc9d27fbfcc347cc9960a4802fb7e8b8', 'request_id': 'req-69d3c56d-cb1c-4690-ae6b-69b090e7559a'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:16.892785 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-69d3c56d-cb1c-4690-ae6b-69b090e7559a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc9d27fbfcc347cc9960a4802fb7e8b8', 'request_id': 'req-69d3c56d-cb1c-4690-ae6b-69b090e7559a'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:16.941849 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-69d3c56d-cb1c-4690-ae6b-69b090e7559a admin admin] The token cache is being invalidate because project dc9d27fbfcc347cc9960a4802fb7e8b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81617) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:14:16.942237 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-69d3c56d-cb1c-4690-ae6b-69b090e7559a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:16.946886 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-69d3c56d-cb1c-4690-ae6b-69b090e7559a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:16.953988 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 900/1798] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:14:16 2022] DELETE /identity/v3/projects/dc9d27fbfcc347cc9960a4802fb7e8b8 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 16:14:16.961642 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dbb6bb-ba08-4561-a851-eef263ece2e1 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:16.962223 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dbb6bb-ba08-4561-a851-eef263ece2e1 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:16.962603 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dbb6bb-ba08-4561-a851-eef263ece2e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:17.041340 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-44dbb6bb-ba08-4561-a851-eef263ece2e1 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:14:17.043919 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-44dbb6bb-ba08-4561-a851-eef263ece2e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:17.300892 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 899/1799] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:14:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:14:17.309892 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24105ab-39a0-4a90-bec2-78d3aa5ba6a9 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:17.310601 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24105ab-39a0-4a90-bec2-78d3aa5ba6a9 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:17.311086 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24105ab-39a0-4a90-bec2-78d3aa5ba6a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:17.363741 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-b24105ab-39a0-4a90-bec2-78d3aa5ba6a9 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:14:17.365411 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-b24105ab-39a0-4a90-bec2-78d3aa5ba6a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:17.388070 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 901/1800] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:14:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:14:17.392886 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dbb6bb-ba08-4561-a851-eef263ece2e1 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:17.408639 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dbb6bb-ba08-4561-a851-eef263ece2e1 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:17.436564 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d14a66-d793-432b-8a25-fc7456d127b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:17.442979 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d14a66-d793-432b-8a25-fc7456d127b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:17.444201 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d14a66-d793-432b-8a25-fc7456d127b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:17.445418 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d14a66-d793-432b-8a25-fc7456d127b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:17.446795 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d14a66-d793-432b-8a25-fc7456d127b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:14:17.449566 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d14a66-d793-432b-8a25-fc7456d127b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:17.459027 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d14a66-d793-432b-8a25-fc7456d127b0 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:17.499979 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 900/1801] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:14:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:14:17.503939 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24105ab-39a0-4a90-bec2-78d3aa5ba6a9 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:17.545997 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24105ab-39a0-4a90-bec2-78d3aa5ba6a9 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:17.562777 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d087342-c0d6-4893-b2b7-c4b8a2040ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:17.564297 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d087342-c0d6-4893-b2b7-c4b8a2040ee4 admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:17.565837 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d087342-c0d6-4893-b2b7-c4b8a2040ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:17.567590 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d087342-c0d6-4893-b2b7-c4b8a2040ee4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:17.571454 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d087342-c0d6-4893-b2b7-c4b8a2040ee4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:17.577134 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d087342-c0d6-4893-b2b7-c4b8a2040ee4 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:17.609570 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:17.668375 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 902/1802] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:14:17 2022] POST /identity/v3/projects => generated 387 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:14:17.673222 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d14a66-d793-432b-8a25-fc7456d127b0 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:17.693499 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d14a66-d793-432b-8a25-fc7456d127b0 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:18.069476 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2e9f91-d8c0-46dc-808b-5df6ae5c7aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:18.073748 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e9f91-d8c0-46dc-808b-5df6ae5c7aba admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:18.074759 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e9f91-d8c0-46dc-808b-5df6ae5c7aba admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:18.075601 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e9f91-d8c0-46dc-808b-5df6ae5c7aba admin admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:18.077896 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2e9f91-d8c0-46dc-808b-5df6ae5c7aba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:18.111205 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2e9f91-d8c0-46dc-808b-5df6ae5c7aba admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:18.252174 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 901/1803] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:14:17 2022] POST /identity/v3/users => generated 294 bytes in 579 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:14:18.262054 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d087342-c0d6-4893-b2b7-c4b8a2040ee4 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:18.367022 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d087342-c0d6-4893-b2b7-c4b8a2040ee4 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:18.388406 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea4f62c-86fd-48b2-b231-d33bdc6e610c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:18.389852 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea4f62c-86fd-48b2-b231-d33bdc6e610c admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:18.390520 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea4f62c-86fd-48b2-b231-d33bdc6e610c admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:18.391313 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea4f62c-86fd-48b2-b231-d33bdc6e610c admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:18.392866 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea4f62c-86fd-48b2-b231-d33bdc6e610c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:18.397757 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea4f62c-86fd-48b2-b231-d33bdc6e610c admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:18.433316 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 903/1804] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:14:18 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:14:18.437417 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2e9f91-d8c0-46dc-808b-5df6ae5c7aba admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:18.456987 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2e9f91-d8c0-46dc-808b-5df6ae5c7aba admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:18.464019 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a7eb9d-6c1a-42d4-94df-fc570e97b18f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:18.465174 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a7eb9d-6c1a-42d4-94df-fc570e97b18f admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:18.465644 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a7eb9d-6c1a-42d4-94df-fc570e97b18f admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:18.466113 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a7eb9d-6c1a-42d4-94df-fc570e97b18f admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:18.467463 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a7eb9d-6c1a-42d4-94df-fc570e97b18f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:18.471494 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a7eb9d-6c1a-42d4-94df-fc570e97b18f admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:18.518960 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 902/1805] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:14:18 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:14:18.524300 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea4f62c-86fd-48b2-b231-d33bdc6e610c admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:18.537527 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea4f62c-86fd-48b2-b231-d33bdc6e610c admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:18.558820 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8297a92e-d286-42d5-8aa2-7daa927436f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:18.560018 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8297a92e-d286-42d5-8aa2-7daa927436f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:18.560833 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8297a92e-d286-42d5-8aa2-7daa927436f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:18.561576 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8297a92e-d286-42d5-8aa2-7daa927436f1 admin admin] PATH_INFO: `/v3/projects/39de8d6c344a4ef4a5dc66bbc070ca99/users/c143f67b234c4baf994ca08ff6a19bc3/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:18.590126 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8297a92e-d286-42d5-8aa2-7daa927436f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39de8d6c344a4ef4a5dc66bbc070ca99, user_id=c143f67b234c4baf994ca08ff6a19bc3, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:18.593736 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8297a92e-d286-42d5-8aa2-7daa927436f1 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:18.618405 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 904/1806] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:14:18 2022] PUT /identity/v3/projects/39de8d6c344a4ef4a5dc66bbc070ca99/users/c143f67b234c4baf994ca08ff6a19bc3/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:14:18.629539 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d511286e-6d88-47b7-b54a-a930b4c2a835 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:18.630085 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d511286e-6d88-47b7-b54a-a930b4c2a835 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:18.630613 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d511286e-6d88-47b7-b54a-a930b4c2a835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:18.686535 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-d511286e-6d88-47b7-b54a-a930b4c2a835 None None] MFA Rules not processed for user `c143f67b234c4baf994ca08ff6a19bc3`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:14:18.688884 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-d511286e-6d88-47b7-b54a-a930b4c2a835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:18.726291 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:18.870661 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 903/1807] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:14:18 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:14:18.871868 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8297a92e-d286-42d5-8aa2-7daa927436f1 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:19.011974 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8297a92e-d286-42d5-8aa2-7daa927436f1 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:19.017968 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce3b3fe-eb31-4dcf-a243-9b2e566dfcfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:19.018746 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce3b3fe-eb31-4dcf-a243-9b2e566dfcfe admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:19.019741 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce3b3fe-eb31-4dcf-a243-9b2e566dfcfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:19.020081 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce3b3fe-eb31-4dcf-a243-9b2e566dfcfe admin admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:19.021050 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce3b3fe-eb31-4dcf-a243-9b2e566dfcfe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:19.023380 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce3b3fe-eb31-4dcf-a243-9b2e566dfcfe admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:19.084249 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 905/1808] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:14:18 2022] POST /identity/v3/projects => generated 385 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 16:14:19.087806 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d511286e-6d88-47b7-b54a-a930b4c2a835 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:19.118969 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d511286e-6d88-47b7-b54a-a930b4c2a835 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:19.240094 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d806c1c3-ec63-489d-9203-ce63e5f19b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:19.240758 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d806c1c3-ec63-489d-9203-ce63e5f19b67 admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:19.241078 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d806c1c3-ec63-489d-9203-ce63e5f19b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:19.241903 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d806c1c3-ec63-489d-9203-ce63e5f19b67 admin admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:19.242742 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d806c1c3-ec63-489d-9203-ce63e5f19b67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:19.245046 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d806c1c3-ec63-489d-9203-ce63e5f19b67 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:19.278961 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 904/1809] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:14:19 2022] POST /identity/v3/users => generated 299 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:14:19.282479 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce3b3fe-eb31-4dcf-a243-9b2e566dfcfe admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:19.326793 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce3b3fe-eb31-4dcf-a243-9b2e566dfcfe admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:19.337195 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a0ccf0-fc01-406f-899a-7e314b5ecd30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:19.338487 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a0ccf0-fc01-406f-899a-7e314b5ecd30 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:19.339294 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a0ccf0-fc01-406f-899a-7e314b5ecd30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:19.340235 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a0ccf0-fc01-406f-899a-7e314b5ecd30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:19.341814 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a0ccf0-fc01-406f-899a-7e314b5ecd30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:19.348167 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a0ccf0-fc01-406f-899a-7e314b5ecd30 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:19.384055 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 906/1810] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:14:19 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 16:14:19.389510 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d806c1c3-ec63-489d-9203-ce63e5f19b67 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:19.406109 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d806c1c3-ec63-489d-9203-ce63e5f19b67 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:19.414293 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8595bbac-173b-4439-b66e-f09683b56031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:19.415291 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8595bbac-173b-4439-b66e-f09683b56031 admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:19.415546 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8595bbac-173b-4439-b66e-f09683b56031 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:19.415874 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8595bbac-173b-4439-b66e-f09683b56031 admin admin] PATH_INFO: `/v3/projects/cf59e33afb5a46bea8dee0010c4bb5f7/users/8c3e2af0476f48259f1e9843e0896f25/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:19.437565 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8595bbac-173b-4439-b66e-f09683b56031 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf59e33afb5a46bea8dee0010c4bb5f7, user_id=8c3e2af0476f48259f1e9843e0896f25, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:19.440693 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8595bbac-173b-4439-b66e-f09683b56031 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:19.460997 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 905/1811] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:14:19 2022] PUT /identity/v3/projects/cf59e33afb5a46bea8dee0010c4bb5f7/users/8c3e2af0476f48259f1e9843e0896f25/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:14:19.469073 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a0ccf0-fc01-406f-899a-7e314b5ecd30 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:19.469577 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8595bbac-173b-4439-b66e-f09683b56031 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:19.472810 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-8595bbac-173b-4439-b66e-f09683b56031 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:19.484439 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a0ccf0-fc01-406f-899a-7e314b5ecd30 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:19.637495 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682b77e1-724e-4115-b22c-834651ce72ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:19.638596 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b77e1-724e-4115-b22c-834651ce72ea admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:19.639430 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b77e1-724e-4115-b22c-834651ce72ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:19.639913 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b77e1-724e-4115-b22c-834651ce72ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:19.641640 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682b77e1-724e-4115-b22c-834651ce72ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:19.645332 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682b77e1-724e-4115-b22c-834651ce72ea admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:19.681667 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 907/1812] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:14:19 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 16:14:19.684356 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682b77e1-724e-4115-b22c-834651ce72ea admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:19.696652 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8595bbac-173b-4439-b66e-f09683b56031 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:19.706149 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682b77e1-724e-4115-b22c-834651ce72ea admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:19.716075 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6089de38-fc68-4bd6-9d62-f9076251b083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:19.718142 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6089de38-fc68-4bd6-9d62-f9076251b083 admin admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:19.718909 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6089de38-fc68-4bd6-9d62-f9076251b083 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:19.719741 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6089de38-fc68-4bd6-9d62-f9076251b083 admin admin] PATH_INFO: `/v3/projects/cf59e33afb5a46bea8dee0010c4bb5f7/users/8c3e2af0476f48259f1e9843e0896f25/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:19.726361 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6089de38-fc68-4bd6-9d62-f9076251b083 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf59e33afb5a46bea8dee0010c4bb5f7, user_id=8c3e2af0476f48259f1e9843e0896f25, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:19.736300 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6089de38-fc68-4bd6-9d62-f9076251b083 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:19.737326 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcd518a-f92d-4ea0-be2d-935decb1108a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:19.744055 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd518a-f92d-4ea0-be2d-935decb1108a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:19.745767 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd518a-f92d-4ea0-be2d-935decb1108a admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:19.746428 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd518a-f92d-4ea0-be2d-935decb1108a admin admin] PATH_INFO: `/v3/projects/359462b10d7b4096b98f543da425b5aa` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:19.751124 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcd518a-f92d-4ea0-be2d-935decb1108a admin admin] RBAC: Authorizing `identity:delete_project(project_id=359462b10d7b4096b98f543da425b5aa)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:19.758465 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcd518a-f92d-4ea0-be2d-935decb1108a admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:19.807520 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 908/1813] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:14:19 2022] PUT /identity/v3/projects/cf59e33afb5a46bea8dee0010c4bb5f7/users/8c3e2af0476f48259f1e9843e0896f25/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:14:19.815666 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e275b4cb-f19f-4689-aaf9-de00832e8706 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:19.816145 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e275b4cb-f19f-4689-aaf9-de00832e8706 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:19.816729 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e275b4cb-f19f-4689-aaf9-de00832e8706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:19.861736 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:19.885751 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-e275b4cb-f19f-4689-aaf9-de00832e8706 None None] MFA Rules not processed for user `8c3e2af0476f48259f1e9843e0896f25`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:14:19.888197 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-e275b4cb-f19f-4689-aaf9-de00832e8706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:19.912374 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:19.975479 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-3bcd518a-f92d-4ea0-be2d-935decb1108a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '359462b10d7b4096b98f543da425b5aa', 'request_id': 'req-3bcd518a-f92d-4ea0-be2d-935decb1108a'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:19.985872 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-3bcd518a-f92d-4ea0-be2d-935decb1108a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '359462b10d7b4096b98f543da425b5aa', 'request_id': 'req-3bcd518a-f92d-4ea0-be2d-935decb1108a'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:19.996179 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-3bcd518a-f92d-4ea0-be2d-935decb1108a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '359462b10d7b4096b98f543da425b5aa', 'request_id': 'req-3bcd518a-f92d-4ea0-be2d-935decb1108a'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:20.018965 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-3bcd518a-f92d-4ea0-be2d-935decb1108a admin admin] The token cache is being invalidate because project 359462b10d7b4096b98f543da425b5aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81618) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:14:20.019346 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-3bcd518a-f92d-4ea0-be2d-935decb1108a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:20.020401 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-3bcd518a-f92d-4ea0-be2d-935decb1108a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:20.024345 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 906/1814] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:14:19 2022] DELETE /identity/v3/projects/359462b10d7b4096b98f543da425b5aa => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:14:20.035516 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcd518a-f92d-4ea0-be2d-935decb1108a admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:20.037564 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-3bcd518a-f92d-4ea0-be2d-935decb1108a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:20.046852 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 909/1815] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:14:19 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:14:20.059806 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e275b4cb-f19f-4689-aaf9-de00832e8706 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:20.062136 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-e275b4cb-f19f-4689-aaf9-de00832e8706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:20.163923 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcd518a-f92d-4ea0-be2d-935decb1108a admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:20.172078 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8143be3d-7a42-404d-b51e-8aef27d1e0dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:20.173120 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8143be3d-7a42-404d-b51e-8aef27d1e0dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:20.173570 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8143be3d-7a42-404d-b51e-8aef27d1e0dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:20.174013 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8143be3d-7a42-404d-b51e-8aef27d1e0dc admin admin] PATH_INFO: `/v3/users/43ed7b1ee54642d49912527d8bcc9a05` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:20.176060 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8143be3d-7a42-404d-b51e-8aef27d1e0dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=43ed7b1ee54642d49912527d8bcc9a05)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:20.180041 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e275b4cb-f19f-4689-aaf9-de00832e8706 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:20.180347 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8143be3d-7a42-404d-b51e-8aef27d1e0dc admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:20.267564 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-8143be3d-7a42-404d-b51e-8aef27d1e0dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43ed7b1ee54642d49912527d8bcc9a05'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:20.284482 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8269a6-7598-41be-b3dc-ef193ed6c236 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:20.285931 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8269a6-7598-41be-b3dc-ef193ed6c236 service cinder] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:20.286185 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8269a6-7598-41be-b3dc-ef193ed6c236 service cinder] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:20.286676 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8269a6-7598-41be-b3dc-ef193ed6c236 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:20.289753 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-8143be3d-7a42-404d-b51e-8aef27d1e0dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43ed7b1ee54642d49912527d8bcc9a05'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:20.338413 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-8143be3d-7a42-404d-b51e-8aef27d1e0dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43ed7b1ee54642d49912527d8bcc9a05'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:20.345354 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-8143be3d-7a42-404d-b51e-8aef27d1e0dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43ed7b1ee54642d49912527d8bcc9a05'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:20.352974 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-8143be3d-7a42-404d-b51e-8aef27d1e0dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43ed7b1ee54642d49912527d8bcc9a05'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:20.363187 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 907/1816] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:14:20 2022] DELETE /identity/v3/users/43ed7b1ee54642d49912527d8bcc9a05 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:14:20.388298 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8269a6-7598-41be-b3dc-ef193ed6c236 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:20.390946 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8269a6-7598-41be-b3dc-ef193ed6c236 service cinder] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:20.408571 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 910/1817] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:14:20 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 16:14:20.954071 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8143be3d-7a42-404d-b51e-8aef27d1e0dc admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:20.955845 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-8143be3d-7a42-404d-b51e-8aef27d1e0dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:21.037788 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:21.213092 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8143be3d-7a42-404d-b51e-8aef27d1e0dc admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:21.240687 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cbe4ef-358a-4e60-81e7-ff0d45668316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:21.242710 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cbe4ef-358a-4e60-81e7-ff0d45668316 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:21.243468 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cbe4ef-358a-4e60-81e7-ff0d45668316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:21.243983 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cbe4ef-358a-4e60-81e7-ff0d45668316 admin admin] PATH_INFO: `/v3/projects/d2aceea319824fffba0aca6673bfc396` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:21.251419 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cbe4ef-358a-4e60-81e7-ff0d45668316 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2aceea319824fffba0aca6673bfc396)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:21.260230 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cbe4ef-358a-4e60-81e7-ff0d45668316 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:21.441695 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-81cbe4ef-358a-4e60-81e7-ff0d45668316 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2aceea319824fffba0aca6673bfc396', 'request_id': 'req-81cbe4ef-358a-4e60-81e7-ff0d45668316'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:21.461724 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-81cbe4ef-358a-4e60-81e7-ff0d45668316 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2aceea319824fffba0aca6673bfc396', 'request_id': 'req-81cbe4ef-358a-4e60-81e7-ff0d45668316'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:21.478923 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-81cbe4ef-358a-4e60-81e7-ff0d45668316 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2aceea319824fffba0aca6673bfc396', 'request_id': 'req-81cbe4ef-358a-4e60-81e7-ff0d45668316'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:21.512032 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-81cbe4ef-358a-4e60-81e7-ff0d45668316 admin admin] The token cache is being invalidate because project d2aceea319824fffba0aca6673bfc396 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81618) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:14:21.512482 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-81cbe4ef-358a-4e60-81e7-ff0d45668316 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:21.513431 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-81cbe4ef-358a-4e60-81e7-ff0d45668316 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:21.519932 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 908/1818] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:14:20 2022] DELETE /identity/v3/projects/d2aceea319824fffba0aca6673bfc396 => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:14:21.521661 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db26a5ca-5d1d-450a-b815-60cd274d397c None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:21.522746 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db26a5ca-5d1d-450a-b815-60cd274d397c None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:21.523818 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db26a5ca-5d1d-450a-b815-60cd274d397c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:21.541466 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:21.562208 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-db26a5ca-5d1d-450a-b815-60cd274d397c None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:14:21.564252 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-db26a5ca-5d1d-450a-b815-60cd274d397c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:21.706080 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 911/1819] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:14:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:14:21.716573 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d27878-7448-4a56-a5e0-d9f81eb9aaa7 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:21.716930 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d27878-7448-4a56-a5e0-d9f81eb9aaa7 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:21.717217 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d27878-7448-4a56-a5e0-d9f81eb9aaa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:21.729662 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db26a5ca-5d1d-450a-b815-60cd274d397c None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:21.731624 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-db26a5ca-5d1d-450a-b815-60cd274d397c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:21.758052 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-b9d27878-7448-4a56-a5e0-d9f81eb9aaa7 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:14:21.763431 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-b9d27878-7448-4a56-a5e0-d9f81eb9aaa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:21.763814 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db26a5ca-5d1d-450a-b815-60cd274d397c None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:21.773726 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d8dbfc-f52a-411e-881c-8476bdc2ac85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:21.774446 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d8dbfc-f52a-411e-881c-8476bdc2ac85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:21.774711 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d8dbfc-f52a-411e-881c-8476bdc2ac85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:21.775550 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d8dbfc-f52a-411e-881c-8476bdc2ac85 admin admin] PATH_INFO: `/v3/users/c143f67b234c4baf994ca08ff6a19bc3` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:21.794356 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d8dbfc-f52a-411e-881c-8476bdc2ac85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c143f67b234c4baf994ca08ff6a19bc3)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:21.796740 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d8dbfc-f52a-411e-881c-8476bdc2ac85 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:21.807165 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 909/1820] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:14:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:14:21.813427 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d27878-7448-4a56-a5e0-d9f81eb9aaa7 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:21.837624 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d27878-7448-4a56-a5e0-d9f81eb9aaa7 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:21.858737 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ced871-68a8-40a5-bd90-71cb2ad3e73f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:21.859797 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ced871-68a8-40a5-bd90-71cb2ad3e73f admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:21.860186 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ced871-68a8-40a5-bd90-71cb2ad3e73f admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:21.861416 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ced871-68a8-40a5-bd90-71cb2ad3e73f admin admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:21.865458 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ced871-68a8-40a5-bd90-71cb2ad3e73f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:14:21.867348 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ced871-68a8-40a5-bd90-71cb2ad3e73f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:21.869118 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ced871-68a8-40a5-bd90-71cb2ad3e73f admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:21.893116 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-22d8dbfc-f52a-411e-881c-8476bdc2ac85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c143f67b234c4baf994ca08ff6a19bc3'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:21.894104 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-22d8dbfc-f52a-411e-881c-8476bdc2ac85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c143f67b234c4baf994ca08ff6a19bc3'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:21.899156 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 910/1821] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:14:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:14:21.904741 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ced871-68a8-40a5-bd90-71cb2ad3e73f admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:21.907682 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-35ced871-68a8-40a5-bd90-71cb2ad3e73f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:21.910425 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-22d8dbfc-f52a-411e-881c-8476bdc2ac85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c143f67b234c4baf994ca08ff6a19bc3'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:21.957096 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ced871-68a8-40a5-bd90-71cb2ad3e73f admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:21.960871 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-22d8dbfc-f52a-411e-881c-8476bdc2ac85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c143f67b234c4baf994ca08ff6a19bc3'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:21.963808 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-22d8dbfc-f52a-411e-881c-8476bdc2ac85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c143f67b234c4baf994ca08ff6a19bc3'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:21.977159 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91330e38-d0a6-43bb-a7fe-8930ef5b970e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:21.979545 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91330e38-d0a6-43bb-a7fe-8930ef5b970e admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:21.985265 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91330e38-d0a6-43bb-a7fe-8930ef5b970e admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:21.986046 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91330e38-d0a6-43bb-a7fe-8930ef5b970e admin admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:21.988376 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91330e38-d0a6-43bb-a7fe-8930ef5b970e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:22.019659 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91330e38-d0a6-43bb-a7fe-8930ef5b970e admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:22.033704 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 912/1822] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:14:21 2022] DELETE /identity/v3/users/c143f67b234c4baf994ca08ff6a19bc3 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:14:22.042926 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf44215-e139-4440-9653-f48c974adc19 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:22.043856 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf44215-e139-4440-9653-f48c974adc19 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:22.045278 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf44215-e139-4440-9653-f48c974adc19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:22.103393 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-fcf44215-e139-4440-9653-f48c974adc19 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:14:22.104879 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-fcf44215-e139-4440-9653-f48c974adc19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:22.173747 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 911/1823] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:14:21 2022] POST /identity/v3/projects => generated 381 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:14:22.180524 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91330e38-d0a6-43bb-a7fe-8930ef5b970e admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:22.430708 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 913/1824] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:14:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:14:22.477181 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91330e38-d0a6-43bb-a7fe-8930ef5b970e admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:22.488881 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3129a5d-adfa-4f1b-91da-44067bb74043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:22.490112 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3129a5d-adfa-4f1b-91da-44067bb74043 admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:22.490633 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3129a5d-adfa-4f1b-91da-44067bb74043 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:22.491023 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3129a5d-adfa-4f1b-91da-44067bb74043 admin admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:22.492348 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3129a5d-adfa-4f1b-91da-44067bb74043 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:22.495033 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3129a5d-adfa-4f1b-91da-44067bb74043 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:22.540236 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 912/1825] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:14:22 2022] POST /identity/v3/users => generated 291 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 16:14:22.545654 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf44215-e139-4440-9653-f48c974adc19 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:22.559447 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf44215-e139-4440-9653-f48c974adc19 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:22.572742 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d278d84-44bf-4e3d-94d7-f01bbe7d532b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:22.578708 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3129a5d-adfa-4f1b-91da-44067bb74043 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:22.582556 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-a3129a5d-adfa-4f1b-91da-44067bb74043 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:22.587691 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d278d84-44bf-4e3d-94d7-f01bbe7d532b admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:22.587842 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d278d84-44bf-4e3d-94d7-f01bbe7d532b admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:22.588119 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d278d84-44bf-4e3d-94d7-f01bbe7d532b admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:22.601537 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d278d84-44bf-4e3d-94d7-f01bbe7d532b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:22.605581 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d278d84-44bf-4e3d-94d7-f01bbe7d532b admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:22.614521 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:22.690672 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 914/1826] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:14:22 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:14:22.709793 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d278d84-44bf-4e3d-94d7-f01bbe7d532b admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:22.758992 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:22.778076 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d278d84-44bf-4e3d-94d7-f01bbe7d532b admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:22.793301 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3129a5d-adfa-4f1b-91da-44067bb74043 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:22.799784 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dfb243-948f-4606-aac7-3d7b05130abb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:22.800842 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dfb243-948f-4606-aac7-3d7b05130abb service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:22.801314 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dfb243-948f-4606-aac7-3d7b05130abb service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:22.802047 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dfb243-948f-4606-aac7-3d7b05130abb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:22.809700 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8287fe-626b-4b22-ba7b-415803eb5d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:22.814024 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8287fe-626b-4b22-ba7b-415803eb5d48 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:22.817991 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dfb243-948f-4606-aac7-3d7b05130abb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:22.818319 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8287fe-626b-4b22-ba7b-415803eb5d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:22.819844 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dfb243-948f-4606-aac7-3d7b05130abb service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:22.825267 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8287fe-626b-4b22-ba7b-415803eb5d48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:22.827836 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8287fe-626b-4b22-ba7b-415803eb5d48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:22.833493 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8287fe-626b-4b22-ba7b-415803eb5d48 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:22.839731 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 913/1827] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:14:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:14:22.873962 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 915/1828] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:14:22 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:14:22.878811 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dfb243-948f-4606-aac7-3d7b05130abb service neutron] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:22.895295 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dfb243-948f-4606-aac7-3d7b05130abb service neutron] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:22.903551 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92d0252-d99f-4e2d-9c51-cbfb2dd57639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:22.904551 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92d0252-d99f-4e2d-9c51-cbfb2dd57639 admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:22.904943 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92d0252-d99f-4e2d-9c51-cbfb2dd57639 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:22.905624 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92d0252-d99f-4e2d-9c51-cbfb2dd57639 admin admin] PATH_INFO: `/v3/projects/de0d009ae18b4acda37f04e8761871bd/users/088543c3814143a4bfbfb697587528fe/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:22.937259 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92d0252-d99f-4e2d-9c51-cbfb2dd57639 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de0d009ae18b4acda37f04e8761871bd, user_id=088543c3814143a4bfbfb697587528fe, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:22.941595 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92d0252-d99f-4e2d-9c51-cbfb2dd57639 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:22.964223 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 914/1829] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:14:22 2022] PUT /identity/v3/projects/de0d009ae18b4acda37f04e8761871bd/users/088543c3814143a4bfbfb697587528fe/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:14:22.978144 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51e733-f1fb-4ac8-9ed0-9bf34c04f6e4 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:22.979202 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51e733-f1fb-4ac8-9ed0-9bf34c04f6e4 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:22.979988 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51e733-f1fb-4ac8-9ed0-9bf34c04f6e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:23.072384 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-3b51e733-f1fb-4ac8-9ed0-9bf34c04f6e4 None None] MFA Rules not processed for user `088543c3814143a4bfbfb697587528fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:14:23.074972 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-3b51e733-f1fb-4ac8-9ed0-9bf34c04f6e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:23.267926 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 916/1830] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:14:22 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:14:23.273072 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92d0252-d99f-4e2d-9c51-cbfb2dd57639 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:23.378915 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92d0252-d99f-4e2d-9c51-cbfb2dd57639 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:23.387173 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c564e4d-f8f0-4380-ae75-35426f90c5c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:23.387864 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c564e4d-f8f0-4380-ae75-35426f90c5c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:23.388187 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c564e4d-f8f0-4380-ae75-35426f90c5c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:23.389207 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c564e4d-f8f0-4380-ae75-35426f90c5c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:23.389982 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c564e4d-f8f0-4380-ae75-35426f90c5c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:23.393712 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c564e4d-f8f0-4380-ae75-35426f90c5c7 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:23.445121 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 915/1831] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:14:23 2022] POST /identity/v3/projects => generated 383 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:14:23.449181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b51e733-f1fb-4ac8-9ed0-9bf34c04f6e4 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:23.469877 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b51e733-f1fb-4ac8-9ed0-9bf34c04f6e4 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:23.581347 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a75e85-579a-4b36-a117-5af75c48a3e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:23.582797 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a75e85-579a-4b36-a117-5af75c48a3e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:23.583740 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a75e85-579a-4b36-a117-5af75c48a3e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:23.583740 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a75e85-579a-4b36-a117-5af75c48a3e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:23.584059 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a75e85-579a-4b36-a117-5af75c48a3e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:23.588156 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a75e85-579a-4b36-a117-5af75c48a3e2 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:23.655914 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 917/1832] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:14:23 2022] POST /identity/v3/users => generated 298 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 16:14:23.656910 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c564e4d-f8f0-4380-ae75-35426f90c5c7 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:23.699470 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c564e4d-f8f0-4380-ae75-35426f90c5c7 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:23.732216 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e559ee-8ca5-4ed0-a8aa-a5c262a507a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:23.742221 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e559ee-8ca5-4ed0-a8aa-a5c262a507a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:23.742727 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e559ee-8ca5-4ed0-a8aa-a5c262a507a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:23.743176 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e559ee-8ca5-4ed0-a8aa-a5c262a507a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:23.745187 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e559ee-8ca5-4ed0-a8aa-a5c262a507a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:23.754565 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e559ee-8ca5-4ed0-a8aa-a5c262a507a2 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:23.772652 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 916/1833] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:14:23 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:14:23.778754 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a75e85-579a-4b36-a117-5af75c48a3e2 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:23.796139 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a75e85-579a-4b36-a117-5af75c48a3e2 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:23.806950 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6df3a8-ea29-4502-922d-afefd2695a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:23.807520 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6df3a8-ea29-4502-922d-afefd2695a57 admin admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:23.807868 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6df3a8-ea29-4502-922d-afefd2695a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:23.808159 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6df3a8-ea29-4502-922d-afefd2695a57 admin admin] PATH_INFO: `/v3/projects/45c36ce2b9a04d60b706b435e5caea7c/users/03c7268e912a467fb8374ac0af38514a/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:23.869043 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6df3a8-ea29-4502-922d-afefd2695a57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45c36ce2b9a04d60b706b435e5caea7c, user_id=03c7268e912a467fb8374ac0af38514a, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:23.873974 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6df3a8-ea29-4502-922d-afefd2695a57 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:23.901392 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 918/1834] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:14:23 2022] PUT /identity/v3/projects/45c36ce2b9a04d60b706b435e5caea7c/users/03c7268e912a467fb8374ac0af38514a/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:14:23.905244 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e559ee-8ca5-4ed0-a8aa-a5c262a507a2 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:23.919200 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e559ee-8ca5-4ed0-a8aa-a5c262a507a2 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:24.018792 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498777dc-3a34-4188-b87c-2d257085844c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:24.019609 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498777dc-3a34-4188-b87c-2d257085844c admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:24.019917 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498777dc-3a34-4188-b87c-2d257085844c admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:24.020761 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498777dc-3a34-4188-b87c-2d257085844c admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:24.020975 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498777dc-3a34-4188-b87c-2d257085844c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:24.035909 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498777dc-3a34-4188-b87c-2d257085844c admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:24.038536 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:24.067862 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6df3a8-ea29-4502-922d-afefd2695a57 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:24.071132 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 917/1835] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:14:23 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 16:14:24.095047 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6df3a8-ea29-4502-922d-afefd2695a57 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:24.105813 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2dac82-7d34-43d1-a303-b4146aed8df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:24.107140 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2dac82-7d34-43d1-a303-b4146aed8df6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:24.107584 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2dac82-7d34-43d1-a303-b4146aed8df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:24.108203 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2dac82-7d34-43d1-a303-b4146aed8df6 admin admin] PATH_INFO: `/v3/projects/45c36ce2b9a04d60b706b435e5caea7c/users/03c7268e912a467fb8374ac0af38514a/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:24.110956 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2dac82-7d34-43d1-a303-b4146aed8df6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45c36ce2b9a04d60b706b435e5caea7c, user_id=03c7268e912a467fb8374ac0af38514a, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:24.114703 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2dac82-7d34-43d1-a303-b4146aed8df6 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:24.129107 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 919/1836] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:14:24 2022] PUT /identity/v3/projects/45c36ce2b9a04d60b706b435e5caea7c/users/03c7268e912a467fb8374ac0af38514a/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:14:24.146418 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e06baa-56a3-45c7-9823-8591632d62ee None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:24.146418 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e06baa-56a3-45c7-9823-8591632d62ee None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:24.146418 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e06baa-56a3-45c7-9823-8591632d62ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:24.207345 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-37e06baa-56a3-45c7-9823-8591632d62ee None None] MFA Rules not processed for user `03c7268e912a467fb8374ac0af38514a`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:14:24.209907 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-37e06baa-56a3-45c7-9823-8591632d62ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:24.276072 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2dac82-7d34-43d1-a303-b4146aed8df6 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:24.279789 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-9f2dac82-7d34-43d1-a303-b4146aed8df6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:24.485678 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:24.521000 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 918/1837] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:14:24 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 377 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:14:24.574190 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2dac82-7d34-43d1-a303-b4146aed8df6 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:24.591822 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f177c2f-cbe1-4e70-97e6-35776bd27be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:24.595016 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f177c2f-cbe1-4e70-97e6-35776bd27be3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:24.595825 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f177c2f-cbe1-4e70-97e6-35776bd27be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:24.596596 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f177c2f-cbe1-4e70-97e6-35776bd27be3 admin admin] PATH_INFO: `/v3/projects/39de8d6c344a4ef4a5dc66bbc070ca99` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:24.599147 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f177c2f-cbe1-4e70-97e6-35776bd27be3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39de8d6c344a4ef4a5dc66bbc070ca99)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:24.603713 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f177c2f-cbe1-4e70-97e6-35776bd27be3 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:24.611820 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e06baa-56a3-45c7-9823-8591632d62ee None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:24.615686 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-37e06baa-56a3-45c7-9823-8591632d62ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:24.798018 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-0f177c2f-cbe1-4e70-97e6-35776bd27be3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39de8d6c344a4ef4a5dc66bbc070ca99', 'request_id': 'req-0f177c2f-cbe1-4e70-97e6-35776bd27be3'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:24.802680 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e06baa-56a3-45c7-9823-8591632d62ee None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:24.823949 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-0f177c2f-cbe1-4e70-97e6-35776bd27be3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39de8d6c344a4ef4a5dc66bbc070ca99', 'request_id': 'req-0f177c2f-cbe1-4e70-97e6-35776bd27be3'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:24.837392 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-0f177c2f-cbe1-4e70-97e6-35776bd27be3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39de8d6c344a4ef4a5dc66bbc070ca99', 'request_id': 'req-0f177c2f-cbe1-4e70-97e6-35776bd27be3'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:24.869195 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-0f177c2f-cbe1-4e70-97e6-35776bd27be3 admin admin] The token cache is being invalidate because project 39de8d6c344a4ef4a5dc66bbc070ca99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81617) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:14:24.869554 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-0f177c2f-cbe1-4e70-97e6-35776bd27be3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:24.872404 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-0f177c2f-cbe1-4e70-97e6-35776bd27be3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:24.876558 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 920/1838] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:14:24 2022] DELETE /identity/v3/projects/39de8d6c344a4ef4a5dc66bbc070ca99 => generated 0 bytes in 603 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 16:14:24.882779 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f177c2f-cbe1-4e70-97e6-35776bd27be3 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:24.889735 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-0f177c2f-cbe1-4e70-97e6-35776bd27be3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:24.968495 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ac3b78-5145-4308-bd68-da7de5bc3f82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:24.969483 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ac3b78-5145-4308-bd68-da7de5bc3f82 service cinder] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:24.970090 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ac3b78-5145-4308-bd68-da7de5bc3f82 service cinder] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:24.971652 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ac3b78-5145-4308-bd68-da7de5bc3f82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:24.974674 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-66ac3b78-5145-4308-bd68-da7de5bc3f82 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:25.091848 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:25.127742 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f177c2f-cbe1-4e70-97e6-35776bd27be3 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:25.134725 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ac3b78-5145-4308-bd68-da7de5bc3f82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:25.136346 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d117c3-8e89-4cb8-83f5-0b2c9603a709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:25.137017 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d117c3-8e89-4cb8-83f5-0b2c9603a709 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:25.137301 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d117c3-8e89-4cb8-83f5-0b2c9603a709 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:25.137571 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d117c3-8e89-4cb8-83f5-0b2c9603a709 admin admin] PATH_INFO: `/v3/users/8c3e2af0476f48259f1e9843e0896f25` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:25.139532 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d117c3-8e89-4cb8-83f5-0b2c9603a709 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c3e2af0476f48259f1e9843e0896f25)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:25.140153 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ac3b78-5145-4308-bd68-da7de5bc3f82 service cinder] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:25.145714 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d117c3-8e89-4cb8-83f5-0b2c9603a709 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:25.168362 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 919/1839] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:14:24 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 16:14:25.222841 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-d0d117c3-8e89-4cb8-83f5-0b2c9603a709 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c3e2af0476f48259f1e9843e0896f25'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:25.223563 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-d0d117c3-8e89-4cb8-83f5-0b2c9603a709 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c3e2af0476f48259f1e9843e0896f25'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:25.242086 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-d0d117c3-8e89-4cb8-83f5-0b2c9603a709 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c3e2af0476f48259f1e9843e0896f25'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:25.267970 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:25.277678 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-d0d117c3-8e89-4cb8-83f5-0b2c9603a709 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c3e2af0476f48259f1e9843e0896f25'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:25.279308 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-d0d117c3-8e89-4cb8-83f5-0b2c9603a709 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c3e2af0476f48259f1e9843e0896f25'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:25.307900 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 921/1840] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:14:24 2022] DELETE /identity/v3/users/8c3e2af0476f48259f1e9843e0896f25 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:14:25.316579 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ac3b78-5145-4308-bd68-da7de5bc3f82 service cinder] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:25.318221 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-66ac3b78-5145-4308-bd68-da7de5bc3f82 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:25.336446 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ac3b78-5145-4308-bd68-da7de5bc3f82 service cinder] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:25.449778 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9619205-f592-4344-a2c4-5895223e14e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:25.450955 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9619205-f592-4344-a2c4-5895223e14e0 service cinder] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:25.451443 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9619205-f592-4344-a2c4-5895223e14e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:25.451967 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9619205-f592-4344-a2c4-5895223e14e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:25.455475 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-e9619205-f592-4344-a2c4-5895223e14e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:25.610522 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9619205-f592-4344-a2c4-5895223e14e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:25.614397 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9619205-f592-4344-a2c4-5895223e14e0 service cinder] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:25.648734 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 920/1841] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:14:25 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 16:14:26.143458 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d117c3-8e89-4cb8-83f5-0b2c9603a709 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:26.145364 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-d0d117c3-8e89-4cb8-83f5-0b2c9603a709 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:26.391123 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d117c3-8e89-4cb8-83f5-0b2c9603a709 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:26.400336 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee6062d-b69b-4ca8-856f-1280e105b773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:26.401769 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee6062d-b69b-4ca8-856f-1280e105b773 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:26.402322 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee6062d-b69b-4ca8-856f-1280e105b773 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:26.402835 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee6062d-b69b-4ca8-856f-1280e105b773 admin admin] PATH_INFO: `/v3/projects/cf59e33afb5a46bea8dee0010c4bb5f7` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:26.405049 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee6062d-b69b-4ca8-856f-1280e105b773 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf59e33afb5a46bea8dee0010c4bb5f7)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:26.409965 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee6062d-b69b-4ca8-856f-1280e105b773 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:26.411939 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9619205-f592-4344-a2c4-5895223e14e0 service cinder] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:26.414599 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-e9619205-f592-4344-a2c4-5895223e14e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:26.443427 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9619205-f592-4344-a2c4-5895223e14e0 service cinder] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:26.450752 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d93b5c2-2ff4-45a8-a266-d0c865f79dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:26.451573 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93b5c2-2ff4-45a8-a266-d0c865f79dee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:26.451953 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93b5c2-2ff4-45a8-a266-d0c865f79dee admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:26.452325 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93b5c2-2ff4-45a8-a266-d0c865f79dee admin admin] PATH_INFO: `/v3/users/088543c3814143a4bfbfb697587528fe` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:26.453743 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d93b5c2-2ff4-45a8-a266-d0c865f79dee admin admin] RBAC: Authorizing `identity:delete_user(user_id=088543c3814143a4bfbfb697587528fe)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:26.456136 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d93b5c2-2ff4-45a8-a266-d0c865f79dee admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:26.530692 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-2d93b5c2-2ff4-45a8-a266-d0c865f79dee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '088543c3814143a4bfbfb697587528fe'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:26.543621 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-2d93b5c2-2ff4-45a8-a266-d0c865f79dee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '088543c3814143a4bfbfb697587528fe'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:26.557955 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-3ee6062d-b69b-4ca8-856f-1280e105b773 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf59e33afb5a46bea8dee0010c4bb5f7', 'request_id': 'req-3ee6062d-b69b-4ca8-856f-1280e105b773'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:26.561625 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-2d93b5c2-2ff4-45a8-a266-d0c865f79dee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '088543c3814143a4bfbfb697587528fe'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:26.576611 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-2d93b5c2-2ff4-45a8-a266-d0c865f79dee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '088543c3814143a4bfbfb697587528fe'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:26.580273 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-2d93b5c2-2ff4-45a8-a266-d0c865f79dee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '088543c3814143a4bfbfb697587528fe'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:26.599923 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-3ee6062d-b69b-4ca8-856f-1280e105b773 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf59e33afb5a46bea8dee0010c4bb5f7', 'request_id': 'req-3ee6062d-b69b-4ca8-856f-1280e105b773'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:26.604108 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 921/1842] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:14:26 2022] DELETE /identity/v3/users/088543c3814143a4bfbfb697587528fe => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:14:26.606229 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bee1b2-407f-4902-984e-be3bb0dd3860 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:26.606647 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bee1b2-407f-4902-984e-be3bb0dd3860 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:26.607898 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bee1b2-407f-4902-984e-be3bb0dd3860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:26.623646 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-3ee6062d-b69b-4ca8-856f-1280e105b773 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf59e33afb5a46bea8dee0010c4bb5f7', 'request_id': 'req-3ee6062d-b69b-4ca8-856f-1280e105b773'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:26.665328 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-3ee6062d-b69b-4ca8-856f-1280e105b773 admin admin] The token cache is being invalidate because project cf59e33afb5a46bea8dee0010c4bb5f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81617) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:14:26.665848 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-3ee6062d-b69b-4ca8-856f-1280e105b773 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:26.669138 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-38bee1b2-407f-4902-984e-be3bb0dd3860 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:14:26.674019 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-38bee1b2-407f-4902-984e-be3bb0dd3860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:26.679671 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-3ee6062d-b69b-4ca8-856f-1280e105b773 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:26.682508 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 922/1843] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:14:26 2022] DELETE /identity/v3/projects/cf59e33afb5a46bea8dee0010c4bb5f7 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:14:26.690010 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbf0f01-68c8-4092-b8ae-f2b88dfb4bab None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:26.690529 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbf0f01-68c8-4092-b8ae-f2b88dfb4bab None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:26.691030 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbf0f01-68c8-4092-b8ae-f2b88dfb4bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:26.699747 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:26.734006 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-3bbf0f01-68c8-4092-b8ae-f2b88dfb4bab None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:14:26.735497 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-3bbf0f01-68c8-4092-b8ae-f2b88dfb4bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:26.838430 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 923/1844] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:14:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:14:26.850441 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6672cc33-8044-4997-b89e-ec187a848ffb None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:26.851763 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6672cc33-8044-4997-b89e-ec187a848ffb None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:26.852276 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6672cc33-8044-4997-b89e-ec187a848ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:26.866761 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 922/1845] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:14:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:14:26.886104 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-6672cc33-8044-4997-b89e-ec187a848ffb None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:14:26.888980 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-6672cc33-8044-4997-b89e-ec187a848ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:26.900638 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 924/1846] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:14:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:14:26.910419 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bee1b2-407f-4902-984e-be3bb0dd3860 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:26.913474 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-38bee1b2-407f-4902-984e-be3bb0dd3860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:26.918813 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6672cc33-8044-4997-b89e-ec187a848ffb None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:26.955753 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6672cc33-8044-4997-b89e-ec187a848ffb None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:26.966934 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01566fcc-4f36-46a6-8384-ec4feb249638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:26.967943 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01566fcc-4f36-46a6-8384-ec4feb249638 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:26.968439 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01566fcc-4f36-46a6-8384-ec4feb249638 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:26.968916 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01566fcc-4f36-46a6-8384-ec4feb249638 admin admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:26.970136 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01566fcc-4f36-46a6-8384-ec4feb249638 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:14:26.970740 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01566fcc-4f36-46a6-8384-ec4feb249638 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:26.974044 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01566fcc-4f36-46a6-8384-ec4feb249638 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:26.980049 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:27.002168 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 925/1847] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:14:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:14:27.008358 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01566fcc-4f36-46a6-8384-ec4feb249638 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:27.041537 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01566fcc-4f36-46a6-8384-ec4feb249638 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:27.053648 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad6edb5-6c1d-4adb-8514-7494e9f2d247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:27.053864 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad6edb5-6c1d-4adb-8514-7494e9f2d247 admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:27.054481 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad6edb5-6c1d-4adb-8514-7494e9f2d247 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:27.055052 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad6edb5-6c1d-4adb-8514-7494e9f2d247 admin admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:27.056589 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad6edb5-6c1d-4adb-8514-7494e9f2d247 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:27.060608 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad6edb5-6c1d-4adb-8514-7494e9f2d247 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:27.122311 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bee1b2-407f-4902-984e-be3bb0dd3860 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:27.139981 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68829723-6c11-4e01-8d41-495011661353 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:27.140308 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 926/1848] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:14:27 2022] POST /identity/v3/projects => generated 385 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:14:27.142096 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad6edb5-6c1d-4adb-8514-7494e9f2d247 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:27.144635 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68829723-6c11-4e01-8d41-495011661353 service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:27.145191 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68829723-6c11-4e01-8d41-495011661353 service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:27.145574 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-7ad6edb5-6c1d-4adb-8514-7494e9f2d247 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:27.146280 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68829723-6c11-4e01-8d41-495011661353 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:27.185882 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68829723-6c11-4e01-8d41-495011661353 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:27.188602 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68829723-6c11-4e01-8d41-495011661353 service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:27.225186 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 923/1849] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:14:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:14:27.228271 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68829723-6c11-4e01-8d41-495011661353 service neutron] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:27.242348 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68829723-6c11-4e01-8d41-495011661353 service neutron] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:27.424793 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7a0e4f-77c5-4bee-b0d9-c19964fd6281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:27.431120 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7a0e4f-77c5-4bee-b0d9-c19964fd6281 admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:27.434539 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7a0e4f-77c5-4bee-b0d9-c19964fd6281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:27.435964 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7a0e4f-77c5-4bee-b0d9-c19964fd6281 admin admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:27.436905 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7a0e4f-77c5-4bee-b0d9-c19964fd6281 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:27.442809 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7a0e4f-77c5-4bee-b0d9-c19964fd6281 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:27.497638 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad6edb5-6c1d-4adb-8514-7494e9f2d247 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:27.507679 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af020b-ac22-49d4-8eaf-c0b74280b89e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:27.509053 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af020b-ac22-49d4-8eaf-c0b74280b89e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:27.510349 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af020b-ac22-49d4-8eaf-c0b74280b89e admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:27.510349 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af020b-ac22-49d4-8eaf-c0b74280b89e admin admin] PATH_INFO: `/v3/users/87108684290945b3abf62784a822fff8` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:27.513096 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73af020b-ac22-49d4-8eaf-c0b74280b89e admin admin] RBAC: Authorizing `identity:delete_user(user_id=87108684290945b3abf62784a822fff8)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:27.513584 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 924/1850] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:14:27 2022] POST /identity/v3/users => generated 293 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:14:27.517221 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73af020b-ac22-49d4-8eaf-c0b74280b89e admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:27.517693 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7a0e4f-77c5-4bee-b0d9-c19964fd6281 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:27.545448 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7a0e4f-77c5-4bee-b0d9-c19964fd6281 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:27.562912 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3c31b6-eac0-4646-a480-1d890271910e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:27.562912 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c31b6-eac0-4646-a480-1d890271910e admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:27.562912 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c31b6-eac0-4646-a480-1d890271910e admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:27.562912 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c31b6-eac0-4646-a480-1d890271910e admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:27.563539 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3c31b6-eac0-4646-a480-1d890271910e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:27.563539 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3c31b6-eac0-4646-a480-1d890271910e admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:27.586149 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 925/1851] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:14:27 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:14:27.590899 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3c31b6-eac0-4646-a480-1d890271910e admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:27.612566 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3c31b6-eac0-4646-a480-1d890271910e admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:27.620008 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b51a020-55d7-4806-9a27-525d1d96c932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:27.621145 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b51a020-55d7-4806-9a27-525d1d96c932 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:27.621678 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b51a020-55d7-4806-9a27-525d1d96c932 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:27.622842 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b51a020-55d7-4806-9a27-525d1d96c932 admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:27.624017 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b51a020-55d7-4806-9a27-525d1d96c932 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:27.626051 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b51a020-55d7-4806-9a27-525d1d96c932 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:27.643826 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 926/1852] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:14:27 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:14:27.649974 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b51a020-55d7-4806-9a27-525d1d96c932 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:27.668668 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b51a020-55d7-4806-9a27-525d1d96c932 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:27.677873 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a699a71a-ea58-45a2-863d-096ee91cca77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:27.679892 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a699a71a-ea58-45a2-863d-096ee91cca77 admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:27.680527 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a699a71a-ea58-45a2-863d-096ee91cca77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:27.680810 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-73af020b-ac22-49d4-8eaf-c0b74280b89e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87108684290945b3abf62784a822fff8'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:27.681233 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a699a71a-ea58-45a2-863d-096ee91cca77 admin admin] PATH_INFO: `/v3/projects/89b98978933a40fc89fcce5a49b8eb19/users/4317922635a24e55ac95183bfb3510be/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:27.682643 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-73af020b-ac22-49d4-8eaf-c0b74280b89e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87108684290945b3abf62784a822fff8'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:27.715268 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a699a71a-ea58-45a2-863d-096ee91cca77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89b98978933a40fc89fcce5a49b8eb19, user_id=4317922635a24e55ac95183bfb3510be, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:27.718715 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-73af020b-ac22-49d4-8eaf-c0b74280b89e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87108684290945b3abf62784a822fff8'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:27.720301 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a699a71a-ea58-45a2-863d-096ee91cca77 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:27.735393 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:27.746632 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 927/1853] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:14:27 2022] PUT /identity/v3/projects/89b98978933a40fc89fcce5a49b8eb19/users/4317922635a24e55ac95183bfb3510be/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:14:27.774990 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1e5eb-334e-4dcd-a9d4-d6b148dacb29 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:27.775803 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1e5eb-334e-4dcd-a9d4-d6b148dacb29 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:27.776169 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1e5eb-334e-4dcd-a9d4-d6b148dacb29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:27.796130 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-73af020b-ac22-49d4-8eaf-c0b74280b89e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87108684290945b3abf62784a822fff8'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:27.798093 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-73af020b-ac22-49d4-8eaf-c0b74280b89e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87108684290945b3abf62784a822fff8'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:27.860382 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 927/1854] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:14:27 2022] DELETE /identity/v3/users/87108684290945b3abf62784a822fff8 => generated 0 bytes in 719 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:14:27.869128 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4149546b-124a-449c-9131-b673c4e42ab8 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:27.869604 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4149546b-124a-449c-9131-b673c4e42ab8 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:27.870093 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4149546b-124a-449c-9131-b673c4e42ab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:27.875537 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-39a1e5eb-334e-4dcd-a9d4-d6b148dacb29 None None] MFA Rules not processed for user `4317922635a24e55ac95183bfb3510be`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:14:27.879250 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-39a1e5eb-334e-4dcd-a9d4-d6b148dacb29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:27.955648 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-4149546b-124a-449c-9131-b673c4e42ab8 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:14:27.957376 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-4149546b-124a-449c-9131-b673c4e42ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:28.041589 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 928/1855] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:14:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:14:28.059837 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a1e5eb-334e-4dcd-a9d4-d6b148dacb29 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:28.061915 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-39a1e5eb-334e-4dcd-a9d4-d6b148dacb29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:28.130434 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:28.180271 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 928/1856] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:14:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:14:28.243473 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4149546b-124a-449c-9131-b673c4e42ab8 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:28.245772 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-4149546b-124a-449c-9131-b673c4e42ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:28.392402 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a1e5eb-334e-4dcd-a9d4-d6b148dacb29 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:28.415101 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe18e4c-a767-4c32-a125-6e3744c64e68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:28.416103 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe18e4c-a767-4c32-a125-6e3744c64e68 service cinder] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:28.416576 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe18e4c-a767-4c32-a125-6e3744c64e68 service cinder] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:28.417032 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe18e4c-a767-4c32-a125-6e3744c64e68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:28.453061 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe18e4c-a767-4c32-a125-6e3744c64e68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:28.455777 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe18e4c-a767-4c32-a125-6e3744c64e68 service cinder] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:28.502760 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4149546b-124a-449c-9131-b673c4e42ab8 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:28.510320 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 929/1857] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:14:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 16:14:28.515366 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30611980-a57f-40c7-8a3d-293349d61131 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:28.516572 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30611980-a57f-40c7-8a3d-293349d61131 service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:28.517112 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30611980-a57f-40c7-8a3d-293349d61131 service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:28.518931 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30611980-a57f-40c7-8a3d-293349d61131 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:28.543925 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30611980-a57f-40c7-8a3d-293349d61131 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:28.547540 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30611980-a57f-40c7-8a3d-293349d61131 service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:28.584089 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 929/1858] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:14:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:14:28.960823 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe18e4c-a767-4c32-a125-6e3744c64e68 service cinder] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:28.963276 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-afe18e4c-a767-4c32-a125-6e3744c64e68 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:29.028730 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe18e4c-a767-4c32-a125-6e3744c64e68 service cinder] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:29.054157 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a02bd3-39e3-4688-91f5-3a006516525d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:29.054854 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a02bd3-39e3-4688-91f5-3a006516525d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:29.055141 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a02bd3-39e3-4688-91f5-3a006516525d admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:29.056500 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a02bd3-39e3-4688-91f5-3a006516525d admin admin] PATH_INFO: `/v3/projects/de0d009ae18b4acda37f04e8761871bd` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:29.056801 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a02bd3-39e3-4688-91f5-3a006516525d admin admin] RBAC: Authorizing `identity:delete_project(project_id=de0d009ae18b4acda37f04e8761871bd)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:29.059241 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a02bd3-39e3-4688-91f5-3a006516525d admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:29.153711 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-f4a02bd3-39e3-4688-91f5-3a006516525d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de0d009ae18b4acda37f04e8761871bd', 'request_id': 'req-f4a02bd3-39e3-4688-91f5-3a006516525d'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:29.175305 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-f4a02bd3-39e3-4688-91f5-3a006516525d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de0d009ae18b4acda37f04e8761871bd', 'request_id': 'req-f4a02bd3-39e3-4688-91f5-3a006516525d'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:29.195131 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-f4a02bd3-39e3-4688-91f5-3a006516525d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de0d009ae18b4acda37f04e8761871bd', 'request_id': 'req-f4a02bd3-39e3-4688-91f5-3a006516525d'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:29.217725 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-f4a02bd3-39e3-4688-91f5-3a006516525d admin admin] The token cache is being invalidate because project de0d009ae18b4acda37f04e8761871bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81618) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:14:29.218195 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-f4a02bd3-39e3-4688-91f5-3a006516525d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:29.220473 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-f4a02bd3-39e3-4688-91f5-3a006516525d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:29.228214 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30611980-a57f-40c7-8a3d-293349d61131 service neutron] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:29.233446 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 930/1859] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:14:28 2022] DELETE /identity/v3/projects/de0d009ae18b4acda37f04e8761871bd => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:14:29.241194 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30611980-a57f-40c7-8a3d-293349d61131 service neutron] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:29.381115 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300d874d-7707-4ae2-97a5-436310e8e031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:29.382870 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300d874d-7707-4ae2-97a5-436310e8e031 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:29.383191 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300d874d-7707-4ae2-97a5-436310e8e031 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:29.383444 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300d874d-7707-4ae2-97a5-436310e8e031 admin admin] PATH_INFO: `/v3/users/03c7268e912a467fb8374ac0af38514a` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:29.384931 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300d874d-7707-4ae2-97a5-436310e8e031 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03c7268e912a467fb8374ac0af38514a)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:29.387281 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300d874d-7707-4ae2-97a5-436310e8e031 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:29.506317 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-300d874d-7707-4ae2-97a5-436310e8e031 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03c7268e912a467fb8374ac0af38514a'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:29.520275 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-300d874d-7707-4ae2-97a5-436310e8e031 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03c7268e912a467fb8374ac0af38514a'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:29.536376 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-300d874d-7707-4ae2-97a5-436310e8e031 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03c7268e912a467fb8374ac0af38514a'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:29.611233 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-300d874d-7707-4ae2-97a5-436310e8e031 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03c7268e912a467fb8374ac0af38514a'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:29.619184 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-300d874d-7707-4ae2-97a5-436310e8e031 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03c7268e912a467fb8374ac0af38514a'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:29.676556 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 930/1860] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:14:29 2022] DELETE /identity/v3/users/03c7268e912a467fb8374ac0af38514a => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:14:29.834919 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a02bd3-39e3-4688-91f5-3a006516525d admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:29.842443 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-f4a02bd3-39e3-4688-91f5-3a006516525d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:29.864025 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a02bd3-39e3-4688-91f5-3a006516525d admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:29.886774 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:29.984936 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5116190d-9450-45fe-88e4-9724ca305b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:29.986821 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5116190d-9450-45fe-88e4-9724ca305b46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:29.988866 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5116190d-9450-45fe-88e4-9724ca305b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:29.990458 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5116190d-9450-45fe-88e4-9724ca305b46 admin admin] PATH_INFO: `/v3/projects/40f231bdad854961a375ce7323ee6255` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:29.992266 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5116190d-9450-45fe-88e4-9724ca305b46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40f231bdad854961a375ce7323ee6255)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:29.995148 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5116190d-9450-45fe-88e4-9724ca305b46 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:30.171919 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-5116190d-9450-45fe-88e4-9724ca305b46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40f231bdad854961a375ce7323ee6255', 'request_id': 'req-5116190d-9450-45fe-88e4-9724ca305b46'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:30.182748 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-5116190d-9450-45fe-88e4-9724ca305b46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40f231bdad854961a375ce7323ee6255', 'request_id': 'req-5116190d-9450-45fe-88e4-9724ca305b46'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:30.191118 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-5116190d-9450-45fe-88e4-9724ca305b46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40f231bdad854961a375ce7323ee6255', 'request_id': 'req-5116190d-9450-45fe-88e4-9724ca305b46'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:30.217776 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-5116190d-9450-45fe-88e4-9724ca305b46 admin admin] The token cache is being invalidate because project 40f231bdad854961a375ce7323ee6255 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81618) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:14:30.218540 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-5116190d-9450-45fe-88e4-9724ca305b46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:30.219500 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-5116190d-9450-45fe-88e4-9724ca305b46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:30.222569 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 931/1861] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:14:29 2022] DELETE /identity/v3/projects/40f231bdad854961a375ce7323ee6255 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:14:30.226691 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300d874d-7707-4ae2-97a5-436310e8e031 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:30.228723 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-300d874d-7707-4ae2-97a5-436310e8e031 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:30.251088 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300d874d-7707-4ae2-97a5-436310e8e031 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:30.269522 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:30.351355 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdeb215-463a-4f84-bd0b-ce57c3abab7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:30.353965 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdeb215-463a-4f84-bd0b-ce57c3abab7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:30.354814 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdeb215-463a-4f84-bd0b-ce57c3abab7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:30.355249 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdeb215-463a-4f84-bd0b-ce57c3abab7d admin admin] PATH_INFO: `/v3/users/f01399cce6c742cdb28e14ab580c3566` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:30.357147 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdeb215-463a-4f84-bd0b-ce57c3abab7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f01399cce6c742cdb28e14ab580c3566)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:30.361327 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdeb215-463a-4f84-bd0b-ce57c3abab7d admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:30.446786 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b3610-1809-4d67-8a99-76d05ff535e7 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:30.447065 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b3610-1809-4d67-8a99-76d05ff535e7 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:30.447689 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b3610-1809-4d67-8a99-76d05ff535e7 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:30.451084 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 932/1862] 10.222.0.57 () {54 vars in 921 bytes} [Mon Sep 19 16:14:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:14:30.458138 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-2fdeb215-463a-4f84-bd0b-ce57c3abab7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f01399cce6c742cdb28e14ab580c3566'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:30.459170 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-2fdeb215-463a-4f84-bd0b-ce57c3abab7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f01399cce6c742cdb28e14ab580c3566'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:30.463072 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d75f21-92ca-4f20-ab35-da2e06d5de1f None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:30.463619 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d75f21-92ca-4f20-ab35-da2e06d5de1f None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:30.464044 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d75f21-92ca-4f20-ab35-da2e06d5de1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:30.473084 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-2fdeb215-463a-4f84-bd0b-ce57c3abab7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f01399cce6c742cdb28e14ab580c3566'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:30.493517 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-31d75f21-92ca-4f20-ab35-da2e06d5de1f None None] MFA Rules not processed for user `1ecd01cc72834b72b5ff2c4b47249737`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:14:30.496313 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-31d75f21-92ca-4f20-ab35-da2e06d5de1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:30.514778 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-2fdeb215-463a-4f84-bd0b-ce57c3abab7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f01399cce6c742cdb28e14ab580c3566'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:30.516024 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-2fdeb215-463a-4f84-bd0b-ce57c3abab7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f01399cce6c742cdb28e14ab580c3566'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:30.554987 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 931/1863] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:14:30 2022] DELETE /identity/v3/users/f01399cce6c742cdb28e14ab580c3566 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:14:30.574766 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdeb215-463a-4f84-bd0b-ce57c3abab7d admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:30.587128 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-2fdeb215-463a-4f84-bd0b-ce57c3abab7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:30.701191 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 933/1864] 10.222.0.57 () {58 vars in 1018 bytes} [Mon Sep 19 16:14:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:14:30.747274 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdeb215-463a-4f84-bd0b-ce57c3abab7d admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:30.753569 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a0542-2b51-436e-b34f-294f2d6ad277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:30.754921 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a0542-2b51-436e-b34f-294f2d6ad277 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:30.755519 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a0542-2b51-436e-b34f-294f2d6ad277 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:30.755965 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a0542-2b51-436e-b34f-294f2d6ad277 admin admin] PATH_INFO: `/v3/projects/45c36ce2b9a04d60b706b435e5caea7c` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:30.758131 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6a0542-2b51-436e-b34f-294f2d6ad277 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45c36ce2b9a04d60b706b435e5caea7c)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:30.760327 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6a0542-2b51-436e-b34f-294f2d6ad277 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:30.849248 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:30.894900 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-cd6a0542-2b51-436e-b34f-294f2d6ad277 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45c36ce2b9a04d60b706b435e5caea7c', 'request_id': 'req-cd6a0542-2b51-436e-b34f-294f2d6ad277'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:30.923042 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-cd6a0542-2b51-436e-b34f-294f2d6ad277 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45c36ce2b9a04d60b706b435e5caea7c', 'request_id': 'req-cd6a0542-2b51-436e-b34f-294f2d6ad277'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:30.938413 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-cd6a0542-2b51-436e-b34f-294f2d6ad277 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45c36ce2b9a04d60b706b435e5caea7c', 'request_id': 'req-cd6a0542-2b51-436e-b34f-294f2d6ad277'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:30.979851 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-cd6a0542-2b51-436e-b34f-294f2d6ad277 admin admin] The token cache is being invalidate because project 45c36ce2b9a04d60b706b435e5caea7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81617) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:14:30.980218 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-cd6a0542-2b51-436e-b34f-294f2d6ad277 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:30.982037 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-cd6a0542-2b51-436e-b34f-294f2d6ad277 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:30.987073 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 932/1865] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:14:30 2022] DELETE /identity/v3/projects/45c36ce2b9a04d60b706b435e5caea7c => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:14:30.995817 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b850bf3-b522-4e9e-83c3-868254b01bab None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:30.996370 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b850bf3-b522-4e9e-83c3-868254b01bab None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:31.002030 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b850bf3-b522-4e9e-83c3-868254b01bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:31.031386 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a0542-2b51-436e-b34f-294f2d6ad277 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:31.039192 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-cd6a0542-2b51-436e-b34f-294f2d6ad277 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:31.056094 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-1b850bf3-b522-4e9e-83c3-868254b01bab None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:14:31.058460 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-1b850bf3-b522-4e9e-83c3-868254b01bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:31.159429 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:31.286738 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 934/1866] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:14:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:14:31.289623 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c0ddf-d695-483a-b1bb-827029e8c050 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:31.289908 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c0ddf-d695-483a-b1bb-827029e8c050 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:31.290181 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c0ddf-d695-483a-b1bb-827029e8c050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:31.294476 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a0542-2b51-436e-b34f-294f2d6ad277 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:31.305678 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5ebb99-9b84-4c55-adbd-7efa67050813 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:31.307028 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ebb99-9b84-4c55-adbd-7efa67050813 service nova] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:31.307574 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ebb99-9b84-4c55-adbd-7efa67050813 service nova] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:31.308153 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ebb99-9b84-4c55-adbd-7efa67050813 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:31.316340 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-fb5ebb99-9b84-4c55-adbd-7efa67050813 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:31.330918 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-753c0ddf-d695-483a-b1bb-827029e8c050 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:14:31.333759 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-753c0ddf-d695-483a-b1bb-827029e8c050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:31.338885 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5ebb99-9b84-4c55-adbd-7efa67050813 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:31.341407 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5ebb99-9b84-4c55-adbd-7efa67050813 service nova] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:31.356735 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 935/1867] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:14:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:14:31.359838 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753c0ddf-d695-483a-b1bb-827029e8c050 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:31.371482 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 933/1868] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:14:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:14:31.386527 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753c0ddf-d695-483a-b1bb-827029e8c050 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:31.389262 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5ebb99-9b84-4c55-adbd-7efa67050813 service nova] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:31.393525 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-fb5ebb99-9b84-4c55-adbd-7efa67050813 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:31.404531 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048d0216-d816-440a-8116-63c1f4c2b46a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:31.409425 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d0216-d816-440a-8116-63c1f4c2b46a admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:31.410551 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d0216-d816-440a-8116-63c1f4c2b46a admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:31.411077 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d0216-d816-440a-8116-63c1f4c2b46a admin admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:31.414826 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048d0216-d816-440a-8116-63c1f4c2b46a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:14:31.415371 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048d0216-d816-440a-8116-63c1f4c2b46a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:31.419762 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5ebb99-9b84-4c55-adbd-7efa67050813 service nova] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:31.427396 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048d0216-d816-440a-8116-63c1f4c2b46a admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:31.433779 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc4c5d2-7694-40d8-aacd-1753bed18ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:31.434935 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc4c5d2-7694-40d8-aacd-1753bed18ec7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:31.435443 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc4c5d2-7694-40d8-aacd-1753bed18ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:31.435936 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc4c5d2-7694-40d8-aacd-1753bed18ec7 admin admin] PATH_INFO: `/v3/projects/ba3247c9fefb4027ac363189468900dd` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:31.439269 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc4c5d2-7694-40d8-aacd-1753bed18ec7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba3247c9fefb4027ac363189468900dd)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:31.443225 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc4c5d2-7694-40d8-aacd-1753bed18ec7 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:31.452378 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 936/1869] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:14:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:14:31.458349 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048d0216-d816-440a-8116-63c1f4c2b46a admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:31.475970 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048d0216-d816-440a-8116-63c1f4c2b46a admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:31.489926 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc26e717-fd9f-4ce8-83e0-e2f7590987af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:31.491816 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc26e717-fd9f-4ce8-83e0-e2f7590987af admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:31.493076 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc26e717-fd9f-4ce8-83e0-e2f7590987af admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:31.493076 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc26e717-fd9f-4ce8-83e0-e2f7590987af admin admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:31.494321 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc26e717-fd9f-4ce8-83e0-e2f7590987af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:31.499541 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc26e717-fd9f-4ce8-83e0-e2f7590987af admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:31.572378 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 937/1870] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:14:31 2022] POST /identity/v3/projects => generated 375 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 16:14:31.573577 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-edc4c5d2-7694-40d8-aacd-1753bed18ec7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba3247c9fefb4027ac363189468900dd', 'request_id': 'req-edc4c5d2-7694-40d8-aacd-1753bed18ec7'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:31.580247 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc26e717-fd9f-4ce8-83e0-e2f7590987af admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:31.583161 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-edc4c5d2-7694-40d8-aacd-1753bed18ec7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba3247c9fefb4027ac363189468900dd', 'request_id': 'req-edc4c5d2-7694-40d8-aacd-1753bed18ec7'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:31.593939 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-edc4c5d2-7694-40d8-aacd-1753bed18ec7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba3247c9fefb4027ac363189468900dd', 'request_id': 'req-edc4c5d2-7694-40d8-aacd-1753bed18ec7'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:31.612315 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-edc4c5d2-7694-40d8-aacd-1753bed18ec7 admin admin] The token cache is being invalidate because project ba3247c9fefb4027ac363189468900dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81617) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:14:31.612659 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-edc4c5d2-7694-40d8-aacd-1753bed18ec7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:31.613371 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-edc4c5d2-7694-40d8-aacd-1753bed18ec7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:31.616604 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 934/1871] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:14:31 2022] DELETE /identity/v3/projects/ba3247c9fefb4027ac363189468900dd => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:14:31.623625 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6675d97a-a9a0-4c87-b96c-1e6f1a65ee7e None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:31.623913 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6675d97a-a9a0-4c87-b96c-1e6f1a65ee7e None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:31.624321 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6675d97a-a9a0-4c87-b96c-1e6f1a65ee7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:31.656707 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-6675d97a-a9a0-4c87-b96c-1e6f1a65ee7e None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:14:31.659205 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-6675d97a-a9a0-4c87-b96c-1e6f1a65ee7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:31.703782 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:31.720116 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc26e717-fd9f-4ce8-83e0-e2f7590987af admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:31.726058 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3de9e27-dda9-43dd-b196-aed198dcacae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:31.726784 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3de9e27-dda9-43dd-b196-aed198dcacae admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:31.727112 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3de9e27-dda9-43dd-b196-aed198dcacae admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:31.727433 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3de9e27-dda9-43dd-b196-aed198dcacae admin admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:31.728130 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3de9e27-dda9-43dd-b196-aed198dcacae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:31.730936 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3de9e27-dda9-43dd-b196-aed198dcacae admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:31.747863 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 935/1872] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:14:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:14:31.756866 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28773d09-419b-4d43-8a0e-0c92220f542b None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:31.757310 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28773d09-419b-4d43-8a0e-0c92220f542b None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:31.757759 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28773d09-419b-4d43-8a0e-0c92220f542b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:31.766093 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 938/1873] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:14:31 2022] POST /identity/v3/users => generated 288 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:14:31.771532 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3de9e27-dda9-43dd-b196-aed198dcacae admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:31.779374 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-a3de9e27-dda9-43dd-b196-aed198dcacae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:31.793985 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-28773d09-419b-4d43-8a0e-0c92220f542b None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:14:31.796339 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-28773d09-419b-4d43-8a0e-0c92220f542b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:31.799782 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3de9e27-dda9-43dd-b196-aed198dcacae admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:31.807880 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc14c39f-ba0a-4a8f-92f9-dd54f891ffee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:31.807880 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc14c39f-ba0a-4a8f-92f9-dd54f891ffee admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:31.807880 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc14c39f-ba0a-4a8f-92f9-dd54f891ffee admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:31.807880 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc14c39f-ba0a-4a8f-92f9-dd54f891ffee admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:31.811744 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc14c39f-ba0a-4a8f-92f9-dd54f891ffee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:31.813693 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc14c39f-ba0a-4a8f-92f9-dd54f891ffee admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:31.829257 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 936/1874] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:14:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:14:31.830908 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28773d09-419b-4d43-8a0e-0c92220f542b None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:31.838728 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 939/1875] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:14:31 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 16:14:31.845789 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc14c39f-ba0a-4a8f-92f9-dd54f891ffee admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:31.852274 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28773d09-419b-4d43-8a0e-0c92220f542b None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:31.859705 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2847b491-adef-4792-b3da-4781cac6b237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:31.860430 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2847b491-adef-4792-b3da-4781cac6b237 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:31.860785 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2847b491-adef-4792-b3da-4781cac6b237 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:31.861125 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2847b491-adef-4792-b3da-4781cac6b237 admin admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:31.862193 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2847b491-adef-4792-b3da-4781cac6b237 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:14:31.862805 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2847b491-adef-4792-b3da-4781cac6b237 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:31.865102 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc14c39f-ba0a-4a8f-92f9-dd54f891ffee admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:31.865557 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2847b491-adef-4792-b3da-4781cac6b237 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:31.875703 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ae3106-d8f3-47ea-a6a5-8b170c7f4ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:31.876947 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae3106-d8f3-47ea-a6a5-8b170c7f4ba1 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:31.877524 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae3106-d8f3-47ea-a6a5-8b170c7f4ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:31.878101 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae3106-d8f3-47ea-a6a5-8b170c7f4ba1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:31.879664 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ae3106-d8f3-47ea-a6a5-8b170c7f4ba1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:31.883410 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 937/1876] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:14:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:14:31.884186 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ae3106-d8f3-47ea-a6a5-8b170c7f4ba1 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:31.887857 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2847b491-adef-4792-b3da-4781cac6b237 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:31.905149 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2847b491-adef-4792-b3da-4781cac6b237 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:31.907598 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 940/1877] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:14:31 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 16:14:31.913674 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2f3827-cc5a-4e9f-9dff-103134c30ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:31.913981 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f3827-cc5a-4e9f-9dff-103134c30ecd admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:31.914640 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f3827-cc5a-4e9f-9dff-103134c30ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:31.915113 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f3827-cc5a-4e9f-9dff-103134c30ecd admin admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:31.916211 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2f3827-cc5a-4e9f-9dff-103134c30ecd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:31.917510 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ae3106-d8f3-47ea-a6a5-8b170c7f4ba1 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:31.920327 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2f3827-cc5a-4e9f-9dff-103134c30ecd admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:31.935718 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ae3106-d8f3-47ea-a6a5-8b170c7f4ba1 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:31.943082 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05b1b8a-f7e8-4c35-9b23-f5b995475512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:31.944757 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b1b8a-f7e8-4c35-9b23-f5b995475512 admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:31.945455 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b1b8a-f7e8-4c35-9b23-f5b995475512 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:31.945820 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b1b8a-f7e8-4c35-9b23-f5b995475512 admin admin] PATH_INFO: `/v3/projects/9b10f274c5ab41acb7351fa0dd22a140/users/9e5e17746cf548a8a9f247e2d96138de/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:31.972265 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05b1b8a-f7e8-4c35-9b23-f5b995475512 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b10f274c5ab41acb7351fa0dd22a140, user_id=9e5e17746cf548a8a9f247e2d96138de, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:31.976070 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05b1b8a-f7e8-4c35-9b23-f5b995475512 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:31.977808 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 938/1878] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:14:31 2022] POST /identity/v3/projects => generated 383 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:14:31.981821 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2f3827-cc5a-4e9f-9dff-103134c30ecd admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:32.002268 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 941/1879] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:14:31 2022] PUT /identity/v3/projects/9b10f274c5ab41acb7351fa0dd22a140/users/9e5e17746cf548a8a9f247e2d96138de/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:14:32.027391 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b3bb6-ce3b-4ad1-96f8-5104375d82bd None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:32.027878 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b3bb6-ce3b-4ad1-96f8-5104375d82bd None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:32.028267 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b3bb6-ce3b-4ad1-96f8-5104375d82bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:32.083684 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-3a5b3bb6-ce3b-4ad1-96f8-5104375d82bd None None] MFA Rules not processed for user `9e5e17746cf548a8a9f247e2d96138de`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:14:32.086132 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-3a5b3bb6-ce3b-4ad1-96f8-5104375d82bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:32.118762 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2f3827-cc5a-4e9f-9dff-103134c30ecd admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:32.148983 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:32.221886 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 942/1880] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:14:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:14:32.239589 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b3bb6-ce3b-4ad1-96f8-5104375d82bd None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:32.242061 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-3a5b3bb6-ce3b-4ad1-96f8-5104375d82bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:32.260973 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d50c31-d6da-4e14-b4d7-b4925bfd43bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:32.262563 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d50c31-d6da-4e14-b4d7-b4925bfd43bf admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:32.263110 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d50c31-d6da-4e14-b4d7-b4925bfd43bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:32.263569 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d50c31-d6da-4e14-b4d7-b4925bfd43bf admin admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:32.264771 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d50c31-d6da-4e14-b4d7-b4925bfd43bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:32.271625 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d50c31-d6da-4e14-b4d7-b4925bfd43bf admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:32.338724 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 939/1881] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:14:31 2022] POST /identity/v3/users => generated 292 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 16:14:32.338724 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d50c31-d6da-4e14-b4d7-b4925bfd43bf admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:32.362405 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d50c31-d6da-4e14-b4d7-b4925bfd43bf admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:32.376873 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef8c4b9-4d9c-40b7-b982-f4983cfa6eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:32.378249 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b3bb6-ce3b-4ad1-96f8-5104375d82bd None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:32.378984 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8c4b9-4d9c-40b7-b982-f4983cfa6eb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:32.380677 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8c4b9-4d9c-40b7-b982-f4983cfa6eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:32.381558 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8c4b9-4d9c-40b7-b982-f4983cfa6eb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:32.383937 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef8c4b9-4d9c-40b7-b982-f4983cfa6eb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:32.384770 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acef8aa9-8f69-4b4c-b54e-909c8c3b6664 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:32.387208 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef8aa9-8f69-4b4c-b54e-909c8c3b6664 service cinder] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:32.387895 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef8aa9-8f69-4b4c-b54e-909c8c3b6664 service cinder] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:32.388172 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef8c4b9-4d9c-40b7-b982-f4983cfa6eb4 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:32.389393 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef8aa9-8f69-4b4c-b54e-909c8c3b6664 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:32.412631 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 940/1882] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:14:32 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:14:32.413345 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acef8aa9-8f69-4b4c-b54e-909c8c3b6664 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:32.415035 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef8c4b9-4d9c-40b7-b982-f4983cfa6eb4 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:32.415825 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acef8aa9-8f69-4b4c-b54e-909c8c3b6664 service cinder] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:32.426723 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef8c4b9-4d9c-40b7-b982-f4983cfa6eb4 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:32.433912 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f4a0e1-4860-44ec-be50-3bb7670c7aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:32.435242 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 943/1883] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:14:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 16:14:32.438573 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4a0e1-4860-44ec-be50-3bb7670c7aa0 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:32.439302 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4a0e1-4860-44ec-be50-3bb7670c7aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:32.440196 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4a0e1-4860-44ec-be50-3bb7670c7aa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:32.441652 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f4a0e1-4860-44ec-be50-3bb7670c7aa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:32.447891 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f4a0e1-4860-44ec-be50-3bb7670c7aa0 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:32.469289 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 941/1884] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:14:32 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 16:14:32.475826 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acef8aa9-8f69-4b4c-b54e-909c8c3b6664 service cinder] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:32.497604 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acef8aa9-8f69-4b4c-b54e-909c8c3b6664 service cinder] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:32.506422 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882907b0-cf06-4af6-868c-fbf2d61d2d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:32.507636 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882907b0-cf06-4af6-868c-fbf2d61d2d2e admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:32.508052 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882907b0-cf06-4af6-868c-fbf2d61d2d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:32.508555 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882907b0-cf06-4af6-868c-fbf2d61d2d2e admin admin] PATH_INFO: `/v3/projects/c937cf9b2db24504bb19a0c22ac28a1d/users/e0a17c74c98d4eb78ca7fa1bbaf2309a/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:32.539856 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882907b0-cf06-4af6-868c-fbf2d61d2d2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c937cf9b2db24504bb19a0c22ac28a1d, user_id=e0a17c74c98d4eb78ca7fa1bbaf2309a, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:32.546911 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882907b0-cf06-4af6-868c-fbf2d61d2d2e admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:32.594858 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 944/1885] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:14:32 2022] PUT /identity/v3/projects/c937cf9b2db24504bb19a0c22ac28a1d/users/e0a17c74c98d4eb78ca7fa1bbaf2309a/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:14:32.600044 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f07655-2a95-4af0-abc2-1ee303d21488 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:32.601568 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f07655-2a95-4af0-abc2-1ee303d21488 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:32.602071 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f07655-2a95-4af0-abc2-1ee303d21488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:32.635722 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:32.658048 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-74f07655-2a95-4af0-abc2-1ee303d21488 None None] MFA Rules not processed for user `e0a17c74c98d4eb78ca7fa1bbaf2309a`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:14:32.659494 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-74f07655-2a95-4af0-abc2-1ee303d21488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:32.796472 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 942/1886] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:14:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:14:32.803117 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882907b0-cf06-4af6-868c-fbf2d61d2d2e admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:32.960622 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882907b0-cf06-4af6-868c-fbf2d61d2d2e admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:32.979755 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34282e27-1c60-43be-b66a-0dbda367d8f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:32.980418 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34282e27-1c60-43be-b66a-0dbda367d8f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:32.980706 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34282e27-1c60-43be-b66a-0dbda367d8f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:32.980987 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34282e27-1c60-43be-b66a-0dbda367d8f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:32.985123 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34282e27-1c60-43be-b66a-0dbda367d8f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:32.987516 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34282e27-1c60-43be-b66a-0dbda367d8f6 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:33.076994 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 945/1887] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:14:32 2022] POST /identity/v3/projects => generated 381 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 16:14:33.078337 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f07655-2a95-4af0-abc2-1ee303d21488 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:33.098255 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f07655-2a95-4af0-abc2-1ee303d21488 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:33.205422 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b17182-278f-4ec2-819d-037cedb28fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:33.206404 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b17182-278f-4ec2-819d-037cedb28fc4 admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:33.206889 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b17182-278f-4ec2-819d-037cedb28fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:33.207366 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b17182-278f-4ec2-819d-037cedb28fc4 admin admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:33.208100 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b17182-278f-4ec2-819d-037cedb28fc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:33.211384 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b17182-278f-4ec2-819d-037cedb28fc4 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:33.247881 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 943/1888] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:14:33 2022] POST /identity/v3/users => generated 297 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:14:33.249437 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34282e27-1c60-43be-b66a-0dbda367d8f6 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:33.261406 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34282e27-1c60-43be-b66a-0dbda367d8f6 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:33.266719 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3f727e-03c5-46a2-bda0-bf448d4bed57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:33.267455 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3f727e-03c5-46a2-bda0-bf448d4bed57 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:33.267918 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3f727e-03c5-46a2-bda0-bf448d4bed57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:33.268065 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3f727e-03c5-46a2-bda0-bf448d4bed57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:33.268658 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3f727e-03c5-46a2-bda0-bf448d4bed57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:33.270404 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3f727e-03c5-46a2-bda0-bf448d4bed57 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:33.282500 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 946/1889] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:14:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:14:33.287057 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b17182-278f-4ec2-819d-037cedb28fc4 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:33.302535 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b17182-278f-4ec2-819d-037cedb28fc4 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:33.307813 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89c9dbc-3dc3-436d-87e0-500c9dfc2eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:33.308495 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89c9dbc-3dc3-436d-87e0-500c9dfc2eac admin admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:33.308774 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89c9dbc-3dc3-436d-87e0-500c9dfc2eac admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:33.309046 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89c9dbc-3dc3-436d-87e0-500c9dfc2eac admin admin] PATH_INFO: `/v3/projects/b702ac388f7446e7b43c54d7bf1003bb/users/9dae757fcdc649ce8bd3ac38837288ae/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:33.330606 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89c9dbc-3dc3-436d-87e0-500c9dfc2eac admin admin] RBAC: Authorizing `identity:create_grant(project_id=b702ac388f7446e7b43c54d7bf1003bb, user_id=9dae757fcdc649ce8bd3ac38837288ae, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:33.333565 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89c9dbc-3dc3-436d-87e0-500c9dfc2eac admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:33.348797 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 944/1890] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:14:33 2022] PUT /identity/v3/projects/b702ac388f7446e7b43c54d7bf1003bb/users/9dae757fcdc649ce8bd3ac38837288ae/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:14:33.354168 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3f727e-03c5-46a2-bda0-bf448d4bed57 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:33.364987 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3f727e-03c5-46a2-bda0-bf448d4bed57 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:33.388042 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:33.438019 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b434457d-24dd-495d-8c7e-c74a1dd7b3dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:33.438699 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b434457d-24dd-495d-8c7e-c74a1dd7b3dd admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:33.438980 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b434457d-24dd-495d-8c7e-c74a1dd7b3dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:33.439255 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b434457d-24dd-495d-8c7e-c74a1dd7b3dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:33.439901 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b434457d-24dd-495d-8c7e-c74a1dd7b3dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:33.441509 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b434457d-24dd-495d-8c7e-c74a1dd7b3dd admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:33.452668 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 947/1891] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:14:33 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:14:33.459793 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89c9dbc-3dc3-436d-87e0-500c9dfc2eac admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:33.470984 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89c9dbc-3dc3-436d-87e0-500c9dfc2eac admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:33.483678 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b4c836-17c5-419d-a0a6-31aa70c191cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:33.484128 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b4c836-17c5-419d-a0a6-31aa70c191cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:33.484576 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b4c836-17c5-419d-a0a6-31aa70c191cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:33.484984 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b4c836-17c5-419d-a0a6-31aa70c191cc admin admin] PATH_INFO: `/v3/projects/b702ac388f7446e7b43c54d7bf1003bb/users/9dae757fcdc649ce8bd3ac38837288ae/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:33.487845 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b4c836-17c5-419d-a0a6-31aa70c191cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b702ac388f7446e7b43c54d7bf1003bb, user_id=9dae757fcdc649ce8bd3ac38837288ae, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:33.490678 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b4c836-17c5-419d-a0a6-31aa70c191cc admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:33.503319 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 945/1892] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:14:33 2022] PUT /identity/v3/projects/b702ac388f7446e7b43c54d7bf1003bb/users/9dae757fcdc649ce8bd3ac38837288ae/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 16:14:33.515272 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f117475e-257b-4336-8b12-b51d853e632c None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:33.515564 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f117475e-257b-4336-8b12-b51d853e632c None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:33.515858 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f117475e-257b-4336-8b12-b51d853e632c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:33.548627 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-f117475e-257b-4336-8b12-b51d853e632c None None] MFA Rules not processed for user `9dae757fcdc649ce8bd3ac38837288ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:14:33.549881 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-f117475e-257b-4336-8b12-b51d853e632c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:33.642714 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 948/1893] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:14:33 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:14:33.655385 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b4c836-17c5-419d-a0a6-31aa70c191cc admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:33.657381 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-b0b4c836-17c5-419d-a0a6-31aa70c191cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:33.731243 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:33.736493 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b4c836-17c5-419d-a0a6-31aa70c191cc admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:33.742031 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf5831-cd93-4d53-85dc-1e9e658f1648 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:33.743020 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf5831-cd93-4d53-85dc-1e9e658f1648 service cinder] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:33.743392 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf5831-cd93-4d53-85dc-1e9e658f1648 service cinder] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:33.744084 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf5831-cd93-4d53-85dc-1e9e658f1648 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:33.755074 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebf5831-cd93-4d53-85dc-1e9e658f1648 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:33.756570 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebf5831-cd93-4d53-85dc-1e9e658f1648 service cinder] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:33.774202 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 946/1894] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:14:33 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:14:33.905071 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f117475e-257b-4336-8b12-b51d853e632c None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:33.915844 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f117475e-257b-4336-8b12-b51d853e632c None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:33.919554 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef266cba-6b71-40d8-a1a2-f3e38ca9a940 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:33.920425 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef266cba-6b71-40d8-a1a2-f3e38ca9a940 service cinder] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:33.920705 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef266cba-6b71-40d8-a1a2-f3e38ca9a940 service cinder] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:33.920978 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef266cba-6b71-40d8-a1a2-f3e38ca9a940 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:33.990730 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef266cba-6b71-40d8-a1a2-f3e38ca9a940 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:33.993568 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef266cba-6b71-40d8-a1a2-f3e38ca9a940 service cinder] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:34.008409 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 949/1895] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:14:33 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:14:39.335914 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf5831-cd93-4d53-85dc-1e9e658f1648 service cinder] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:39.486225 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf5831-cd93-4d53-85dc-1e9e658f1648 service cinder] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:39.498084 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f060769-66ad-427c-a141-651c64ba8adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:39.498822 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f060769-66ad-427c-a141-651c64ba8adb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:39.499169 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f060769-66ad-427c-a141-651c64ba8adb admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:39.499425 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f060769-66ad-427c-a141-651c64ba8adb admin admin] PATH_INFO: `/v3/users/e0a17c74c98d4eb78ca7fa1bbaf2309a` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:39.501073 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f060769-66ad-427c-a141-651c64ba8adb admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0a17c74c98d4eb78ca7fa1bbaf2309a)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:39.503846 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f060769-66ad-427c-a141-651c64ba8adb admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:39.600830 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-7f060769-66ad-427c-a141-651c64ba8adb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0a17c74c98d4eb78ca7fa1bbaf2309a'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:39.601887 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-7f060769-66ad-427c-a141-651c64ba8adb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0a17c74c98d4eb78ca7fa1bbaf2309a'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:39.616791 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-7f060769-66ad-427c-a141-651c64ba8adb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0a17c74c98d4eb78ca7fa1bbaf2309a'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:39.647665 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-7f060769-66ad-427c-a141-651c64ba8adb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0a17c74c98d4eb78ca7fa1bbaf2309a'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:39.649400 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-7f060769-66ad-427c-a141-651c64ba8adb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0a17c74c98d4eb78ca7fa1bbaf2309a'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:39.696903 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 947/1896] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:14:39 2022] DELETE /identity/v3/users/e0a17c74c98d4eb78ca7fa1bbaf2309a => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:14:39.701483 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34cb298-9963-46ce-b3d7-b211ee240a2f None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:39.702576 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34cb298-9963-46ce-b3d7-b211ee240a2f None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:39.703442 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34cb298-9963-46ce-b3d7-b211ee240a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:39.767994 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-a34cb298-9963-46ce-b3d7-b211ee240a2f None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:14:39.769268 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-a34cb298-9963-46ce-b3d7-b211ee240a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:39.858215 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:39.897910 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 950/1897] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:14:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:14:39.911808 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f060769-66ad-427c-a141-651c64ba8adb admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:39.913464 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-7f060769-66ad-427c-a141-651c64ba8adb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:39.995463 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f060769-66ad-427c-a141-651c64ba8adb admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:40.001580 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109adb73-891e-414e-b50c-3a935354df87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:40.002540 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109adb73-891e-414e-b50c-3a935354df87 service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:40.003006 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109adb73-891e-414e-b50c-3a935354df87 service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:40.003456 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109adb73-891e-414e-b50c-3a935354df87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:40.019737 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109adb73-891e-414e-b50c-3a935354df87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:40.021464 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109adb73-891e-414e-b50c-3a935354df87 service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:40.034246 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 948/1898] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:14:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:14:40.514402 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34cb298-9963-46ce-b3d7-b211ee240a2f None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:40.517066 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-a34cb298-9963-46ce-b3d7-b211ee240a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:40.542705 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34cb298-9963-46ce-b3d7-b211ee240a2f None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:40.549482 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2426922a-a3b9-4fa3-8e1b-2ec6049da9f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:40.550339 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426922a-a3b9-4fa3-8e1b-2ec6049da9f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:40.550746 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426922a-a3b9-4fa3-8e1b-2ec6049da9f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:40.551077 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426922a-a3b9-4fa3-8e1b-2ec6049da9f0 admin admin] PATH_INFO: `/v3/projects/c937cf9b2db24504bb19a0c22ac28a1d` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:40.552839 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2426922a-a3b9-4fa3-8e1b-2ec6049da9f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c937cf9b2db24504bb19a0c22ac28a1d)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:40.554753 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2426922a-a3b9-4fa3-8e1b-2ec6049da9f0 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:40.616083 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-2426922a-a3b9-4fa3-8e1b-2ec6049da9f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c937cf9b2db24504bb19a0c22ac28a1d', 'request_id': 'req-2426922a-a3b9-4fa3-8e1b-2ec6049da9f0'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:40.624396 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-2426922a-a3b9-4fa3-8e1b-2ec6049da9f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c937cf9b2db24504bb19a0c22ac28a1d', 'request_id': 'req-2426922a-a3b9-4fa3-8e1b-2ec6049da9f0'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:40.631312 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-2426922a-a3b9-4fa3-8e1b-2ec6049da9f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c937cf9b2db24504bb19a0c22ac28a1d', 'request_id': 'req-2426922a-a3b9-4fa3-8e1b-2ec6049da9f0'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:40.649758 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-2426922a-a3b9-4fa3-8e1b-2ec6049da9f0 admin admin] The token cache is being invalidate because project c937cf9b2db24504bb19a0c22ac28a1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81618) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:14:40.650153 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-2426922a-a3b9-4fa3-8e1b-2ec6049da9f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:40.651059 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-2426922a-a3b9-4fa3-8e1b-2ec6049da9f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:40.653091 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 951/1899] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:14:40 2022] DELETE /identity/v3/projects/c937cf9b2db24504bb19a0c22ac28a1d => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:14:40.657167 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109adb73-891e-414e-b50c-3a935354df87 service neutron] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:40.668639 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109adb73-891e-414e-b50c-3a935354df87 service neutron] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:40.680774 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:40.762895 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1909b18-87ec-4b86-b353-bf177f2a9094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:40.764009 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1909b18-87ec-4b86-b353-bf177f2a9094 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:40.764563 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1909b18-87ec-4b86-b353-bf177f2a9094 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:40.764957 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1909b18-87ec-4b86-b353-bf177f2a9094 admin admin] PATH_INFO: `/v3/users/9dae757fcdc649ce8bd3ac38837288ae` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:40.767394 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1909b18-87ec-4b86-b353-bf177f2a9094 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9dae757fcdc649ce8bd3ac38837288ae)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:40.771083 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1909b18-87ec-4b86-b353-bf177f2a9094 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:40.833522 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-a1909b18-87ec-4b86-b353-bf177f2a9094 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9dae757fcdc649ce8bd3ac38837288ae'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:40.834478 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-a1909b18-87ec-4b86-b353-bf177f2a9094 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9dae757fcdc649ce8bd3ac38837288ae'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:40.845604 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-a1909b18-87ec-4b86-b353-bf177f2a9094 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9dae757fcdc649ce8bd3ac38837288ae'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:40.860460 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-a1909b18-87ec-4b86-b353-bf177f2a9094 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9dae757fcdc649ce8bd3ac38837288ae'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:40.861225 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-a1909b18-87ec-4b86-b353-bf177f2a9094 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9dae757fcdc649ce8bd3ac38837288ae'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:40.877253 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 949/1900] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:14:40 2022] DELETE /identity/v3/users/9dae757fcdc649ce8bd3ac38837288ae => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 16:14:41.454428 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2426922a-a3b9-4fa3-8e1b-2ec6049da9f0 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:41.456050 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-2426922a-a3b9-4fa3-8e1b-2ec6049da9f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:41.471661 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2426922a-a3b9-4fa3-8e1b-2ec6049da9f0 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:41.570366 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5ca45f-d04e-409f-861c-e65255f1d975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:41.571170 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5ca45f-d04e-409f-861c-e65255f1d975 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:41.571495 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5ca45f-d04e-409f-861c-e65255f1d975 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:41.571809 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5ca45f-d04e-409f-861c-e65255f1d975 admin admin] PATH_INFO: `/v3/projects/b702ac388f7446e7b43c54d7bf1003bb` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:41.573374 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5ca45f-d04e-409f-861c-e65255f1d975 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b702ac388f7446e7b43c54d7bf1003bb)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:41.576046 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5ca45f-d04e-409f-861c-e65255f1d975 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:41.655239 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-0a5ca45f-d04e-409f-861c-e65255f1d975 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b702ac388f7446e7b43c54d7bf1003bb', 'request_id': 'req-0a5ca45f-d04e-409f-861c-e65255f1d975'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:41.663462 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-0a5ca45f-d04e-409f-861c-e65255f1d975 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b702ac388f7446e7b43c54d7bf1003bb', 'request_id': 'req-0a5ca45f-d04e-409f-861c-e65255f1d975'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:41.672060 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-0a5ca45f-d04e-409f-861c-e65255f1d975 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b702ac388f7446e7b43c54d7bf1003bb', 'request_id': 'req-0a5ca45f-d04e-409f-861c-e65255f1d975'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:41.676640 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:41.693798 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-0a5ca45f-d04e-409f-861c-e65255f1d975 admin admin] The token cache is being invalidate because project b702ac388f7446e7b43c54d7bf1003bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81618) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:14:41.694464 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-0a5ca45f-d04e-409f-861c-e65255f1d975 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:41.695631 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-0a5ca45f-d04e-409f-861c-e65255f1d975 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:41.697713 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 952/1901] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:14:41 2022] DELETE /identity/v3/projects/b702ac388f7446e7b43c54d7bf1003bb => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 16:14:41.703248 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05152f57-863a-4790-977d-f9b9a0f02824 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:41.704086 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05152f57-863a-4790-977d-f9b9a0f02824 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:41.704086 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05152f57-863a-4790-977d-f9b9a0f02824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:41.725510 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-05152f57-863a-4790-977d-f9b9a0f02824 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:14:41.727203 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-05152f57-863a-4790-977d-f9b9a0f02824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:41.813899 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 950/1902] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:14:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:14:41.826799 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c153c91-2fe7-4849-9eea-c0ecb9f235c5 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:41.827109 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c153c91-2fe7-4849-9eea-c0ecb9f235c5 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:41.827423 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c153c91-2fe7-4849-9eea-c0ecb9f235c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:41.849164 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-1c153c91-2fe7-4849-9eea-c0ecb9f235c5 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:14:41.851115 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-1c153c91-2fe7-4849-9eea-c0ecb9f235c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:41.859742 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 953/1903] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:14:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:14:41.865404 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05152f57-863a-4790-977d-f9b9a0f02824 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:41.876424 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05152f57-863a-4790-977d-f9b9a0f02824 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:41.882195 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d38e181-9ab0-457c-a2c1-253c5c8a0d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:41.883077 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d38e181-9ab0-457c-a2c1-253c5c8a0d12 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:41.883404 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d38e181-9ab0-457c-a2c1-253c5c8a0d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:41.883718 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d38e181-9ab0-457c-a2c1-253c5c8a0d12 admin admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:41.884687 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d38e181-9ab0-457c-a2c1-253c5c8a0d12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:14:41.884936 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d38e181-9ab0-457c-a2c1-253c5c8a0d12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:41.887124 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d38e181-9ab0-457c-a2c1-253c5c8a0d12 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:41.907403 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 951/1904] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:14:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:14:41.913313 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c153c91-2fe7-4849-9eea-c0ecb9f235c5 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:41.925244 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c153c91-2fe7-4849-9eea-c0ecb9f235c5 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:41.930175 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed7acef-39c1-4ffe-ae72-e3a775b9dfc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:41.930840 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed7acef-39c1-4ffe-ae72-e3a775b9dfc3 admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:41.931119 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed7acef-39c1-4ffe-ae72-e3a775b9dfc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:41.931389 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed7acef-39c1-4ffe-ae72-e3a775b9dfc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:41.932046 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed7acef-39c1-4ffe-ae72-e3a775b9dfc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:41.933995 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed7acef-39c1-4ffe-ae72-e3a775b9dfc3 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:41.970516 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 954/1905] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:14:41 2022] POST /identity/v3/projects => generated 371 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 16:14:41.976816 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d38e181-9ab0-457c-a2c1-253c5c8a0d12 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:41.989573 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d38e181-9ab0-457c-a2c1-253c5c8a0d12 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:42.020649 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:42.106182 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45825cc1-7b93-400a-b1a3-63f7b9bce73e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:42.107743 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45825cc1-7b93-400a-b1a3-63f7b9bce73e admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:42.108859 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45825cc1-7b93-400a-b1a3-63f7b9bce73e admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:42.109410 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45825cc1-7b93-400a-b1a3-63f7b9bce73e admin admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:42.111062 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45825cc1-7b93-400a-b1a3-63f7b9bce73e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:42.115097 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45825cc1-7b93-400a-b1a3-63f7b9bce73e admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:42.158419 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 952/1906] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:14:41 2022] POST /identity/v3/users => generated 286 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:14:42.163898 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed7acef-39c1-4ffe-ae72-e3a775b9dfc3 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:42.181278 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed7acef-39c1-4ffe-ae72-e3a775b9dfc3 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:42.189456 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74919b7b-8e21-4de2-a84b-c6038ae96e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:42.190641 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74919b7b-8e21-4de2-a84b-c6038ae96e2d admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:42.191136 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74919b7b-8e21-4de2-a84b-c6038ae96e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:42.191634 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74919b7b-8e21-4de2-a84b-c6038ae96e2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:42.192807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74919b7b-8e21-4de2-a84b-c6038ae96e2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:42.196324 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74919b7b-8e21-4de2-a84b-c6038ae96e2d admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:42.214675 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 955/1907] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:14:42.220233 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45825cc1-7b93-400a-b1a3-63f7b9bce73e admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:42.240533 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45825cc1-7b93-400a-b1a3-63f7b9bce73e admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:42.249192 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff1ba78-9fbb-4dcf-88d0-5db19d148d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:42.250013 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff1ba78-9fbb-4dcf-88d0-5db19d148d9b admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:42.250524 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff1ba78-9fbb-4dcf-88d0-5db19d148d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:42.250923 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff1ba78-9fbb-4dcf-88d0-5db19d148d9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:42.251871 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff1ba78-9fbb-4dcf-88d0-5db19d148d9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:42.254198 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff1ba78-9fbb-4dcf-88d0-5db19d148d9b admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:42.284986 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 953/1908] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:14:42.291453 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74919b7b-8e21-4de2-a84b-c6038ae96e2d admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:42.339598 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74919b7b-8e21-4de2-a84b-c6038ae96e2d admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:42.382754 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef29ef7b-715b-49f0-b1bd-d4e6d9af8a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:42.387498 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef29ef7b-715b-49f0-b1bd-d4e6d9af8a3a admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:42.393531 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef29ef7b-715b-49f0-b1bd-d4e6d9af8a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:42.398420 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef29ef7b-715b-49f0-b1bd-d4e6d9af8a3a admin admin] PATH_INFO: `/v3/projects/15abe68f64cd487590d31fb8eabd28ae/users/04ec95dde7c245e598d6ca82f476eb49/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:42.475381 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef29ef7b-715b-49f0-b1bd-d4e6d9af8a3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=15abe68f64cd487590d31fb8eabd28ae, user_id=04ec95dde7c245e598d6ca82f476eb49, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:42.482859 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef29ef7b-715b-49f0-b1bd-d4e6d9af8a3a admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:42.524105 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 956/1909] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:14:42 2022] PUT /identity/v3/projects/15abe68f64cd487590d31fb8eabd28ae/users/04ec95dde7c245e598d6ca82f476eb49/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:14:42.534980 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d1155-9d65-4d00-a247-e702dad17c9f None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:42.535614 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d1155-9d65-4d00-a247-e702dad17c9f None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:42.536231 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d1155-9d65-4d00-a247-e702dad17c9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:42.594803 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-b21d1155-9d65-4d00-a247-e702dad17c9f None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:14:42.599107 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-b21d1155-9d65-4d00-a247-e702dad17c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:42.616716 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef29ef7b-715b-49f0-b1bd-d4e6d9af8a3a admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:42.620690 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-ef29ef7b-715b-49f0-b1bd-d4e6d9af8a3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:42.703586 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 954/1910] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:14:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:14:42.718241 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21d1155-9d65-4d00-a247-e702dad17c9f None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:42.721943 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-b21d1155-9d65-4d00-a247-e702dad17c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:42.844179 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21d1155-9d65-4d00-a247-e702dad17c9f None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:42.853474 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88611d47-b187-4497-9f02-b838859fd508 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:42.854160 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88611d47-b187-4497-9f02-b838859fd508 service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:42.854480 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88611d47-b187-4497-9f02-b838859fd508 service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:42.854759 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88611d47-b187-4497-9f02-b838859fd508 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:42.870343 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef29ef7b-715b-49f0-b1bd-d4e6d9af8a3a admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:42.872568 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88611d47-b187-4497-9f02-b838859fd508 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:42.874234 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88611d47-b187-4497-9f02-b838859fd508 service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:42.880761 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7d1d94-f194-47ad-9e39-9b8a1e63348a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:42.882809 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d1d94-f194-47ad-9e39-9b8a1e63348a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:42.883338 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d1d94-f194-47ad-9e39-9b8a1e63348a admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:42.884869 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d1d94-f194-47ad-9e39-9b8a1e63348a admin admin] PATH_INFO: `/v3/users/4317922635a24e55ac95183bfb3510be` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:42.887786 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7d1d94-f194-47ad-9e39-9b8a1e63348a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4317922635a24e55ac95183bfb3510be)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:42.892045 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7d1d94-f194-47ad-9e39-9b8a1e63348a admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:42.895730 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 955/1911] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:14:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:14:42.974006 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-dc7d1d94-f194-47ad-9e39-9b8a1e63348a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4317922635a24e55ac95183bfb3510be'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:43.005940 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-dc7d1d94-f194-47ad-9e39-9b8a1e63348a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4317922635a24e55ac95183bfb3510be'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:43.040900 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-dc7d1d94-f194-47ad-9e39-9b8a1e63348a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4317922635a24e55ac95183bfb3510be'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:43.055270 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-dc7d1d94-f194-47ad-9e39-9b8a1e63348a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4317922635a24e55ac95183bfb3510be'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:43.056824 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-dc7d1d94-f194-47ad-9e39-9b8a1e63348a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4317922635a24e55ac95183bfb3510be'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:43.070033 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 957/1912] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:14:42 2022] DELETE /identity/v3/users/4317922635a24e55ac95183bfb3510be => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:14:43.075770 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3994a1-0179-49bc-ad4f-6f5e65a91d38 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:43.076209 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3994a1-0179-49bc-ad4f-6f5e65a91d38 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:43.076682 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3994a1-0179-49bc-ad4f-6f5e65a91d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:43.118810 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-8e3994a1-0179-49bc-ad4f-6f5e65a91d38 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:14:43.122081 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-8e3994a1-0179-49bc-ad4f-6f5e65a91d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:43.233021 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:43.263675 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 956/1913] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:14:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:14:43.294492 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7d1d94-f194-47ad-9e39-9b8a1e63348a admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:43.298607 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-dc7d1d94-f194-47ad-9e39-9b8a1e63348a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:43.332338 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:43.438490 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7d1d94-f194-47ad-9e39-9b8a1e63348a admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:43.451950 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cf27d8-ccf2-403e-a231-f93d10ddd722 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:43.453414 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cf27d8-ccf2-403e-a231-f93d10ddd722 service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:43.453972 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cf27d8-ccf2-403e-a231-f93d10ddd722 service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:43.454517 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cf27d8-ccf2-403e-a231-f93d10ddd722 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:43.476143 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cf27d8-ccf2-403e-a231-f93d10ddd722 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:43.477620 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cf27d8-ccf2-403e-a231-f93d10ddd722 service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:43.489622 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 958/1914] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:14:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 16:14:44.172523 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3994a1-0179-49bc-ad4f-6f5e65a91d38 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:44.175145 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-8e3994a1-0179-49bc-ad4f-6f5e65a91d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:44.208719 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3994a1-0179-49bc-ad4f-6f5e65a91d38 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:44.235538 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016313d6-3dc4-47fc-b71d-40100d4714ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:44.236220 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016313d6-3dc4-47fc-b71d-40100d4714ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:44.236863 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016313d6-3dc4-47fc-b71d-40100d4714ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:44.237143 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016313d6-3dc4-47fc-b71d-40100d4714ee admin admin] PATH_INFO: `/v3/projects/89b98978933a40fc89fcce5a49b8eb19` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:44.243005 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016313d6-3dc4-47fc-b71d-40100d4714ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=89b98978933a40fc89fcce5a49b8eb19)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:44.245420 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016313d6-3dc4-47fc-b71d-40100d4714ee admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:44.384982 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-016313d6-3dc4-47fc-b71d-40100d4714ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89b98978933a40fc89fcce5a49b8eb19', 'request_id': 'req-016313d6-3dc4-47fc-b71d-40100d4714ee'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:44.403917 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-016313d6-3dc4-47fc-b71d-40100d4714ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89b98978933a40fc89fcce5a49b8eb19', 'request_id': 'req-016313d6-3dc4-47fc-b71d-40100d4714ee'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:44.417761 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-016313d6-3dc4-47fc-b71d-40100d4714ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89b98978933a40fc89fcce5a49b8eb19', 'request_id': 'req-016313d6-3dc4-47fc-b71d-40100d4714ee'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:44.460877 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-016313d6-3dc4-47fc-b71d-40100d4714ee admin admin] The token cache is being invalidate because project 89b98978933a40fc89fcce5a49b8eb19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81617) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:14:44.461181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-016313d6-3dc4-47fc-b71d-40100d4714ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:44.465367 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-016313d6-3dc4-47fc-b71d-40100d4714ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:44.472185 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 957/1915] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:14:44 2022] DELETE /identity/v3/projects/89b98978933a40fc89fcce5a49b8eb19 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:14:44.475081 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb220a0-d3f9-413e-a9a6-8b635a90c254 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:44.475764 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb220a0-d3f9-413e-a9a6-8b635a90c254 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:44.476313 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb220a0-d3f9-413e-a9a6-8b635a90c254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:44.530442 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-ceb220a0-d3f9-413e-a9a6-8b635a90c254 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:14:44.533320 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-ceb220a0-d3f9-413e-a9a6-8b635a90c254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:44.895320 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 959/1916] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:14:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:14:44.931368 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c1efb-c55e-4124-a11d-8ae0df6dd8cc None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:44.933914 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c1efb-c55e-4124-a11d-8ae0df6dd8cc None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:44.934872 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c1efb-c55e-4124-a11d-8ae0df6dd8cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:45.026712 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-e51c1efb-c55e-4124-a11d-8ae0df6dd8cc None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:14:45.030129 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-e51c1efb-c55e-4124-a11d-8ae0df6dd8cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:45.050804 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 958/1917] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:14:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:14:45.068368 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb220a0-d3f9-413e-a9a6-8b635a90c254 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:45.088503 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb220a0-d3f9-413e-a9a6-8b635a90c254 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:45.108020 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836f1e2c-0e14-4e00-b56f-4c3c33266d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:45.110184 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836f1e2c-0e14-4e00-b56f-4c3c33266d48 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:45.110697 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836f1e2c-0e14-4e00-b56f-4c3c33266d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:45.111320 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836f1e2c-0e14-4e00-b56f-4c3c33266d48 admin admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:45.112774 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836f1e2c-0e14-4e00-b56f-4c3c33266d48 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:14:45.113504 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836f1e2c-0e14-4e00-b56f-4c3c33266d48 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:45.117254 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836f1e2c-0e14-4e00-b56f-4c3c33266d48 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:45.141818 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 960/1918] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:14:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:14:45.147391 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51c1efb-c55e-4124-a11d-8ae0df6dd8cc None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:45.164960 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f592cccf-15b0-4b33-8e3e-75151a312bb0 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:45.165553 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f592cccf-15b0-4b33-8e3e-75151a312bb0 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:45.166088 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f592cccf-15b0-4b33-8e3e-75151a312bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:45.174252 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51c1efb-c55e-4124-a11d-8ae0df6dd8cc None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:45.192824 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067032d3-f529-4e00-87ae-edcfeeace3be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:45.194317 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067032d3-f529-4e00-87ae-edcfeeace3be admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:45.194741 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067032d3-f529-4e00-87ae-edcfeeace3be admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:45.195137 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067032d3-f529-4e00-87ae-edcfeeace3be admin admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:45.195935 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067032d3-f529-4e00-87ae-edcfeeace3be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:45.202118 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067032d3-f529-4e00-87ae-edcfeeace3be admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:45.248773 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 959/1919] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:14:45 2022] POST /identity/v3/projects => generated 371 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:14:45.252826 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067032d3-f529-4e00-87ae-edcfeeace3be admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:45.288520 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-f592cccf-15b0-4b33-8e3e-75151a312bb0 None None] MFA Rules not processed for user `04ec95dde7c245e598d6ca82f476eb49`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:14:45.290578 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-f592cccf-15b0-4b33-8e3e-75151a312bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:45.414870 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067032d3-f529-4e00-87ae-edcfeeace3be admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:45.423721 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8b3cf7-ca03-4bf6-84ed-c9ef0a1ace78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:45.424979 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8b3cf7-ca03-4bf6-84ed-c9ef0a1ace78 admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:45.425598 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8b3cf7-ca03-4bf6-84ed-c9ef0a1ace78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:45.426085 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8b3cf7-ca03-4bf6-84ed-c9ef0a1ace78 admin admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:45.427334 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8b3cf7-ca03-4bf6-84ed-c9ef0a1ace78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:45.432390 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8b3cf7-ca03-4bf6-84ed-c9ef0a1ace78 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:45.459967 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 961/1920] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:14:45 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:14:45.463808 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f592cccf-15b0-4b33-8e3e-75151a312bb0 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:45.466503 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-f592cccf-15b0-4b33-8e3e-75151a312bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:45.494674 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 960/1921] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:14:45 2022] POST /identity/v3/users => generated 286 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 16:14:45.496882 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8b3cf7-ca03-4bf6-84ed-c9ef0a1ace78 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:45.499538 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f592cccf-15b0-4b33-8e3e-75151a312bb0 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:45.507120 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4de53e3-027b-48e3-ae9a-2432a4dce291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:45.508076 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de53e3-027b-48e3-ae9a-2432a4dce291 admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:45.508510 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de53e3-027b-48e3-ae9a-2432a4dce291 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:45.509051 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de53e3-027b-48e3-ae9a-2432a4dce291 admin admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:45.510034 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4de53e3-027b-48e3-ae9a-2432a4dce291 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:45.512878 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4de53e3-027b-48e3-ae9a-2432a4dce291 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:45.520203 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8b3cf7-ca03-4bf6-84ed-c9ef0a1ace78 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:45.531762 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ce82cc-e3b8-410b-a8e3-dab6d5e84938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:45.533281 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce82cc-e3b8-410b-a8e3-dab6d5e84938 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:45.534298 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce82cc-e3b8-410b-a8e3-dab6d5e84938 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:45.534933 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce82cc-e3b8-410b-a8e3-dab6d5e84938 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:45.536307 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ce82cc-e3b8-410b-a8e3-dab6d5e84938 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:45.540262 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ce82cc-e3b8-410b-a8e3-dab6d5e84938 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:45.570438 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 961/1922] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:14:45 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:14:45.571865 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 962/1923] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:14:45 2022] POST /identity/v3/projects => generated 373 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 16:14:45.575389 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ce82cc-e3b8-410b-a8e3-dab6d5e84938 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:45.577031 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4de53e3-027b-48e3-ae9a-2432a4dce291 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:45.599001 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ce82cc-e3b8-410b-a8e3-dab6d5e84938 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:45.602193 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:45.622743 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:45.694737 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3a2e52-1b5c-4d9a-a2d1-1ef19c1b9c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:45.695839 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3a2e52-1b5c-4d9a-a2d1-1ef19c1b9c10 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:45.696411 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3a2e52-1b5c-4d9a-a2d1-1ef19c1b9c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:45.696846 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3a2e52-1b5c-4d9a-a2d1-1ef19c1b9c10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:45.697765 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3a2e52-1b5c-4d9a-a2d1-1ef19c1b9c10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:45.699816 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3a2e52-1b5c-4d9a-a2d1-1ef19c1b9c10 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:45.754635 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 962/1924] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:14:45 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:14:45.757236 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4de53e3-027b-48e3-ae9a-2432a4dce291 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:45.763792 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3a2e52-1b5c-4d9a-a2d1-1ef19c1b9c10 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:45.780023 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4412737d-6c31-4044-908b-aef22cfe3513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:45.780884 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412737d-6c31-4044-908b-aef22cfe3513 admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:45.782200 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412737d-6c31-4044-908b-aef22cfe3513 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:45.782524 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412737d-6c31-4044-908b-aef22cfe3513 admin admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:45.784380 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4412737d-6c31-4044-908b-aef22cfe3513 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:45.790248 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4412737d-6c31-4044-908b-aef22cfe3513 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:45.808308 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3a2e52-1b5c-4d9a-a2d1-1ef19c1b9c10 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:45.823554 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9296934-c1f0-4b8a-8044-d2aa940fffce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:45.826483 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9296934-c1f0-4b8a-8044-d2aa940fffce admin admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:45.826483 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9296934-c1f0-4b8a-8044-d2aa940fffce admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:45.830323 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9296934-c1f0-4b8a-8044-d2aa940fffce admin admin] PATH_INFO: `/v3/projects/0e641d48680849328816261ffd5b3b42/users/fe870f5b19334b47b972d7655876472e/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:45.843789 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 963/1925] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:14:45 2022] POST /identity/v3/users => generated 293 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:14:45.847697 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4412737d-6c31-4044-908b-aef22cfe3513 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:45.864418 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9296934-c1f0-4b8a-8044-d2aa940fffce admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e641d48680849328816261ffd5b3b42, user_id=fe870f5b19334b47b972d7655876472e, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:45.868395 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9296934-c1f0-4b8a-8044-d2aa940fffce admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:45.869497 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4412737d-6c31-4044-908b-aef22cfe3513 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:45.908487 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f750ea0-1024-4746-b620-cbd5d78b8975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:45.909509 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 963/1926] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:14:45 2022] PUT /identity/v3/projects/0e641d48680849328816261ffd5b3b42/users/fe870f5b19334b47b972d7655876472e/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:14:45.909943 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f750ea0-1024-4746-b620-cbd5d78b8975 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:45.910644 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f750ea0-1024-4746-b620-cbd5d78b8975 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:45.911453 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f750ea0-1024-4746-b620-cbd5d78b8975 admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:45.912003 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f750ea0-1024-4746-b620-cbd5d78b8975 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:45.915646 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f750ea0-1024-4746-b620-cbd5d78b8975 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:45.921055 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c819c9b4-8203-4a45-9f13-cc69b534fdf4 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:45.925716 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c819c9b4-8203-4a45-9f13-cc69b534fdf4 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:45.926200 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c819c9b4-8203-4a45-9f13-cc69b534fdf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:45.945493 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 964/1927] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:14:45 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 16:14:45.952865 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f750ea0-1024-4746-b620-cbd5d78b8975 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:45.966669 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-c819c9b4-8203-4a45-9f13-cc69b534fdf4 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:14:45.966669 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-c819c9b4-8203-4a45-9f13-cc69b534fdf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:45.983325 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f750ea0-1024-4746-b620-cbd5d78b8975 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:46.091859 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 964/1928] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:14:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:14:46.107247 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c819c9b4-8203-4a45-9f13-cc69b534fdf4 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:46.109303 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-c819c9b4-8203-4a45-9f13-cc69b534fdf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:46.148707 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17fc62f-9e3a-4820-8e6f-dea0b517aa5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:46.149556 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17fc62f-9e3a-4820-8e6f-dea0b517aa5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:46.149945 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17fc62f-9e3a-4820-8e6f-dea0b517aa5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:46.150326 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17fc62f-9e3a-4820-8e6f-dea0b517aa5d admin admin] PATH_INFO: `/v3/projects/b4a885a9387e4d50a1ad3ef26419a909/users/218615224abf468c85c11a3d97914ee4/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:46.177385 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17fc62f-9e3a-4820-8e6f-dea0b517aa5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4a885a9387e4d50a1ad3ef26419a909, user_id=218615224abf468c85c11a3d97914ee4, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:46.184122 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17fc62f-9e3a-4820-8e6f-dea0b517aa5d admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:46.208789 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 965/1929] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:14:45 2022] PUT /identity/v3/projects/b4a885a9387e4d50a1ad3ef26419a909/users/218615224abf468c85c11a3d97914ee4/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:14:46.214439 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17fc62f-9e3a-4820-8e6f-dea0b517aa5d admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:46.281911 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c819c9b4-8203-4a45-9f13-cc69b534fdf4 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:46.372457 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17fc62f-9e3a-4820-8e6f-dea0b517aa5d admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:46.407242 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3b71f3-30a2-408e-a58c-617c16e96944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:46.408456 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b71f3-30a2-408e-a58c-617c16e96944 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:46.408706 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b71f3-30a2-408e-a58c-617c16e96944 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:46.408923 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b71f3-30a2-408e-a58c-617c16e96944 admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:46.410172 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3b71f3-30a2-408e-a58c-617c16e96944 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:46.413703 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3b71f3-30a2-408e-a58c-617c16e96944 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:46.432097 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:46.433358 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 966/1930] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:14:46 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:14:46.436486 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3b71f3-30a2-408e-a58c-617c16e96944 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:46.442708 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895a4f31-ce25-43cf-9494-bc7480d7a654 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:46.445339 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a4f31-ce25-43cf-9494-bc7480d7a654 service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:46.445759 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a4f31-ce25-43cf-9494-bc7480d7a654 service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:46.446141 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a4f31-ce25-43cf-9494-bc7480d7a654 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:46.462927 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895a4f31-ce25-43cf-9494-bc7480d7a654 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:46.463098 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3b71f3-30a2-408e-a58c-617c16e96944 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:46.463258 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895a4f31-ce25-43cf-9494-bc7480d7a654 service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:46.471877 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c290a0-a8df-4861-9192-679f58b478bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:46.473103 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c290a0-a8df-4861-9192-679f58b478bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:46.473590 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c290a0-a8df-4861-9192-679f58b478bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:46.475948 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c290a0-a8df-4861-9192-679f58b478bf admin admin] PATH_INFO: `/v3/projects/b4a885a9387e4d50a1ad3ef26419a909/users/218615224abf468c85c11a3d97914ee4/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:46.476146 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 965/1931] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:14:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:14:46.478822 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c290a0-a8df-4861-9192-679f58b478bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4a885a9387e4d50a1ad3ef26419a909, user_id=218615224abf468c85c11a3d97914ee4, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:46.484158 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c290a0-a8df-4861-9192-679f58b478bf admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:46.488907 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:46.525340 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 967/1932] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:14:46 2022] PUT /identity/v3/projects/b4a885a9387e4d50a1ad3ef26419a909/users/218615224abf468c85c11a3d97914ee4/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:14:47.512838 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dbe5f4-0b9f-49b5-9866-598cf43718a3 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:47.513697 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dbe5f4-0b9f-49b5-9866-598cf43718a3 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:47.513697 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dbe5f4-0b9f-49b5-9866-598cf43718a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:47.573707 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-62dbe5f4-0b9f-49b5-9866-598cf43718a3 None None] MFA Rules not processed for user `fe870f5b19334b47b972d7655876472e`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:14:47.575491 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-62dbe5f4-0b9f-49b5-9866-598cf43718a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:47.740803 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 966/1933] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:14:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:14:47.755391 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c290a0-a8df-4861-9192-679f58b478bf admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:47.758657 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-92c290a0-a8df-4861-9192-679f58b478bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:47.800991 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67eb0a8-2039-4517-833e-25b89dac4d04 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:47.802413 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67eb0a8-2039-4517-833e-25b89dac4d04 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:47.802413 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67eb0a8-2039-4517-833e-25b89dac4d04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:47.887014 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c290a0-a8df-4861-9192-679f58b478bf admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:47.892164 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-a67eb0a8-2039-4517-833e-25b89dac4d04 None None] MFA Rules not processed for user `218615224abf468c85c11a3d97914ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:14:47.893685 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-a67eb0a8-2039-4517-833e-25b89dac4d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:47.896362 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7d380e-c512-448b-b292-90038336fc9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:47.897759 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7d380e-c512-448b-b292-90038336fc9a service cinder] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:47.898552 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7d380e-c512-448b-b292-90038336fc9a service cinder] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:47.899217 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7d380e-c512-448b-b292-90038336fc9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:47.917392 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7d380e-c512-448b-b292-90038336fc9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:47.919760 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7d380e-c512-448b-b292-90038336fc9a service cinder] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:47.934645 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 968/1934] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:14:47 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:14:48.084152 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 967/1935] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:14:47 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:14:48.101555 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7d380e-c512-448b-b292-90038336fc9a service cinder] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:48.103924 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-0d7d380e-c512-448b-b292-90038336fc9a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:48.122812 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7d380e-c512-448b-b292-90038336fc9a service cinder] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:48.136318 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6b7816-6020-4faf-9f31-1fd16018e8a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:48.137267 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6b7816-6020-4faf-9f31-1fd16018e8a9 service cinder] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:48.137690 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6b7816-6020-4faf-9f31-1fd16018e8a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:48.138054 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6b7816-6020-4faf-9f31-1fd16018e8a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:48.243998 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6b7816-6020-4faf-9f31-1fd16018e8a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:48.247782 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6b7816-6020-4faf-9f31-1fd16018e8a9 service cinder] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:48.269804 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 969/1936] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:14:48 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:14:50.051461 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67eb0a8-2039-4517-833e-25b89dac4d04 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:50.070859 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67eb0a8-2039-4517-833e-25b89dac4d04 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:50.077560 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c84c027-168c-4967-be8c-2a01d58c3fe8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:50.079708 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84c027-168c-4967-be8c-2a01d58c3fe8 service cinder] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:50.079708 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84c027-168c-4967-be8c-2a01d58c3fe8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:50.079708 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84c027-168c-4967-be8c-2a01d58c3fe8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:50.102295 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c84c027-168c-4967-be8c-2a01d58c3fe8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:50.109537 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c84c027-168c-4967-be8c-2a01d58c3fe8 service cinder] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:50.141124 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 968/1937] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:14:50 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:14:50.765914 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e084bd-4797-4cf0-b311-df0cd9315756 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:50.766749 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e084bd-4797-4cf0-b311-df0cd9315756 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:50.766749 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e084bd-4797-4cf0-b311-df0cd9315756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:50.804434 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-74e084bd-4797-4cf0-b311-df0cd9315756 None None] MFA Rules not processed for user `fe870f5b19334b47b972d7655876472e`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:14:50.806819 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-74e084bd-4797-4cf0-b311-df0cd9315756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:50.841699 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 970/1938] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:14:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:14:54.471248 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c84c027-168c-4967-be8c-2a01d58c3fe8 service cinder] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:54.473347 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-0c84c027-168c-4967-be8c-2a01d58c3fe8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:54.558910 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:54.578866 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c84c027-168c-4967-be8c-2a01d58c3fe8 service cinder] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:54.585844 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789d3454-de0c-4abd-8c21-b6e1e9e769f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:54.587078 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789d3454-de0c-4abd-8c21-b6e1e9e769f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:54.587479 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789d3454-de0c-4abd-8c21-b6e1e9e769f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:54.587864 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789d3454-de0c-4abd-8c21-b6e1e9e769f0 admin admin] PATH_INFO: `/v3/users/9e5e17746cf548a8a9f247e2d96138de` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:54.589455 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789d3454-de0c-4abd-8c21-b6e1e9e769f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e5e17746cf548a8a9f247e2d96138de)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:54.592395 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789d3454-de0c-4abd-8c21-b6e1e9e769f0 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:54.657771 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-789d3454-de0c-4abd-8c21-b6e1e9e769f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e5e17746cf548a8a9f247e2d96138de'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:54.659032 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-789d3454-de0c-4abd-8c21-b6e1e9e769f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e5e17746cf548a8a9f247e2d96138de'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:54.686749 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-789d3454-de0c-4abd-8c21-b6e1e9e769f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e5e17746cf548a8a9f247e2d96138de'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:54.703455 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-789d3454-de0c-4abd-8c21-b6e1e9e769f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e5e17746cf548a8a9f247e2d96138de'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:54.704375 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-789d3454-de0c-4abd-8c21-b6e1e9e769f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e5e17746cf548a8a9f247e2d96138de'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:54.744050 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 969/1939] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:14:54 2022] DELETE /identity/v3/users/9e5e17746cf548a8a9f247e2d96138de => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:14:54.750308 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bb847e-0668-4f1e-a4e5-bfabf28347be None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:54.751181 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bb847e-0668-4f1e-a4e5-bfabf28347be None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:54.751181 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bb847e-0668-4f1e-a4e5-bfabf28347be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:54.786309 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-96bb847e-0668-4f1e-a4e5-bfabf28347be None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:14:54.789213 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-96bb847e-0668-4f1e-a4e5-bfabf28347be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:54.891460 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 971/1940] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:14:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:14:54.906360 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789d3454-de0c-4abd-8c21-b6e1e9e769f0 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:54.908913 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-789d3454-de0c-4abd-8c21-b6e1e9e769f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:55.008161 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789d3454-de0c-4abd-8c21-b6e1e9e769f0 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:55.014844 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91beba1b-0272-4159-8af0-349b8e618c0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:55.015915 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91beba1b-0272-4159-8af0-349b8e618c0c service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:55.016401 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91beba1b-0272-4159-8af0-349b8e618c0c service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:55.016945 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91beba1b-0272-4159-8af0-349b8e618c0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:55.038979 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91beba1b-0272-4159-8af0-349b8e618c0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:55.042264 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91beba1b-0272-4159-8af0-349b8e618c0c service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:55.063293 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 970/1941] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:14:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:14:55.935203 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bb847e-0668-4f1e-a4e5-bfabf28347be None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:55.939890 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-96bb847e-0668-4f1e-a4e5-bfabf28347be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:55.964048 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bb847e-0668-4f1e-a4e5-bfabf28347be None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:55.972039 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0264ddf8-230e-4b9d-9e9b-fb71149da987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:55.973269 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0264ddf8-230e-4b9d-9e9b-fb71149da987 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:55.973904 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0264ddf8-230e-4b9d-9e9b-fb71149da987 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:55.975028 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0264ddf8-230e-4b9d-9e9b-fb71149da987 admin admin] PATH_INFO: `/v3/projects/9b10f274c5ab41acb7351fa0dd22a140` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:55.976998 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0264ddf8-230e-4b9d-9e9b-fb71149da987 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b10f274c5ab41acb7351fa0dd22a140)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:55.979815 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0264ddf8-230e-4b9d-9e9b-fb71149da987 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:56.020810 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:INSERT=1,keystone:SELECT=98,keystone:DELETE=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:56.069104 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-0264ddf8-230e-4b9d-9e9b-fb71149da987 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b10f274c5ab41acb7351fa0dd22a140', 'request_id': 'req-0264ddf8-230e-4b9d-9e9b-fb71149da987'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:56.081109 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-0264ddf8-230e-4b9d-9e9b-fb71149da987 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b10f274c5ab41acb7351fa0dd22a140', 'request_id': 'req-0264ddf8-230e-4b9d-9e9b-fb71149da987'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:56.092670 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-0264ddf8-230e-4b9d-9e9b-fb71149da987 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b10f274c5ab41acb7351fa0dd22a140', 'request_id': 'req-0264ddf8-230e-4b9d-9e9b-fb71149da987'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:56.117121 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-0264ddf8-230e-4b9d-9e9b-fb71149da987 admin admin] The token cache is being invalidate because project 9b10f274c5ab41acb7351fa0dd22a140 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81618) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:14:56.117792 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-0264ddf8-230e-4b9d-9e9b-fb71149da987 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:56.119207 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-0264ddf8-230e-4b9d-9e9b-fb71149da987 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:14:56.122729 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 972/1942] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:14:55 2022] DELETE /identity/v3/projects/9b10f274c5ab41acb7351fa0dd22a140 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:14:56.128028 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dfc182-36e3-40b5-86cb-b9828eb42e14 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:56.128712 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dfc182-36e3-40b5-86cb-b9828eb42e14 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:56.129006 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dfc182-36e3-40b5-86cb-b9828eb42e14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:56.157881 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-c0dfc182-36e3-40b5-86cb-b9828eb42e14 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:14:56.159933 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-c0dfc182-36e3-40b5-86cb-b9828eb42e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:56.284967 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 971/1943] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:14:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:14:56.295339 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc757572-f22a-48cd-a204-ca65b25dc694 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:56.296140 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc757572-f22a-48cd-a204-ca65b25dc694 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:56.296711 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc757572-f22a-48cd-a204-ca65b25dc694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:56.339222 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-bc757572-f22a-48cd-a204-ca65b25dc694 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:14:56.342005 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-bc757572-f22a-48cd-a204-ca65b25dc694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:56.357275 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 973/1944] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:14:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:14:56.361390 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dfc182-36e3-40b5-86cb-b9828eb42e14 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:56.377740 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dfc182-36e3-40b5-86cb-b9828eb42e14 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:56.384899 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f69d46a-09e4-4cec-b26b-040840a6166f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:56.385536 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f69d46a-09e4-4cec-b26b-040840a6166f admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:56.385821 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f69d46a-09e4-4cec-b26b-040840a6166f admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:56.386092 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f69d46a-09e4-4cec-b26b-040840a6166f admin admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:56.386862 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f69d46a-09e4-4cec-b26b-040840a6166f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:14:56.387170 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f69d46a-09e4-4cec-b26b-040840a6166f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:56.388875 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f69d46a-09e4-4cec-b26b-040840a6166f admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:56.398426 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:56.410233 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 972/1945] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:14:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:14:56.411227 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc757572-f22a-48cd-a204-ca65b25dc694 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:56.425251 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc757572-f22a-48cd-a204-ca65b25dc694 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:56.431889 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d61ed51-c96e-4272-9921-9e1d2dc7b7db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:56.433022 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d61ed51-c96e-4272-9921-9e1d2dc7b7db admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:56.433022 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d61ed51-c96e-4272-9921-9e1d2dc7b7db admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:56.433563 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d61ed51-c96e-4272-9921-9e1d2dc7b7db admin admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:56.435354 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d61ed51-c96e-4272-9921-9e1d2dc7b7db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:56.437547 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d61ed51-c96e-4272-9921-9e1d2dc7b7db admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:56.480370 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 974/1946] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:14:56 2022] POST /identity/v3/projects => generated 375 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 16:14:56.485365 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f69d46a-09e4-4cec-b26b-040840a6166f admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:56.503592 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f69d46a-09e4-4cec-b26b-040840a6166f admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:56.749829 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5920d466-a5df-42f6-98db-d5ecc6e6d28f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:56.751092 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5920d466-a5df-42f6-98db-d5ecc6e6d28f admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:56.751599 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5920d466-a5df-42f6-98db-d5ecc6e6d28f admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:56.752096 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5920d466-a5df-42f6-98db-d5ecc6e6d28f admin admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:56.753306 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5920d466-a5df-42f6-98db-d5ecc6e6d28f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:56.757062 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5920d466-a5df-42f6-98db-d5ecc6e6d28f admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:56.890451 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 973/1947] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:14:56 2022] POST /identity/v3/users => generated 288 bytes in 406 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:14:56.890950 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d61ed51-c96e-4272-9921-9e1d2dc7b7db admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:56.910241 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d61ed51-c96e-4272-9921-9e1d2dc7b7db admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:56.923356 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d5bbef-647e-4401-89e3-f175de15bc27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:56.924205 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d5bbef-647e-4401-89e3-f175de15bc27 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:56.924596 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d5bbef-647e-4401-89e3-f175de15bc27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:56.924990 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d5bbef-647e-4401-89e3-f175de15bc27 admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:56.931416 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d5bbef-647e-4401-89e3-f175de15bc27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:56.933913 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d5bbef-647e-4401-89e3-f175de15bc27 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:56.961716 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5920d466-a5df-42f6-98db-d5ecc6e6d28f admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:56.961991 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 975/1948] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:14:56 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:14:56.974900 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5920d466-a5df-42f6-98db-d5ecc6e6d28f admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:56.989305 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e155ec46-3e9d-4f1d-b2cd-b9afcd9b6170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:56.990540 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e155ec46-3e9d-4f1d-b2cd-b9afcd9b6170 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:56.990946 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e155ec46-3e9d-4f1d-b2cd-b9afcd9b6170 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:56.999308 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e155ec46-3e9d-4f1d-b2cd-b9afcd9b6170 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:57.003459 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e155ec46-3e9d-4f1d-b2cd-b9afcd9b6170 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:57.013748 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e155ec46-3e9d-4f1d-b2cd-b9afcd9b6170 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:57.053060 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 974/1949] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:14:56 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 16:14:57.059162 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d5bbef-647e-4401-89e3-f175de15bc27 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:57.088605 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d5bbef-647e-4401-89e3-f175de15bc27 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:57.106242 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7ee4a0-6add-4b08-a21a-69ffcad19290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:57.107273 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ee4a0-6add-4b08-a21a-69ffcad19290 admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:57.107273 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ee4a0-6add-4b08-a21a-69ffcad19290 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:57.107695 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ee4a0-6add-4b08-a21a-69ffcad19290 admin admin] PATH_INFO: `/v3/projects/7d7da3c84e0746c6accded1b14cf4de6/users/a0449380d9894626b5ead93cf60e96da/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:57.148151 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7ee4a0-6add-4b08-a21a-69ffcad19290 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d7da3c84e0746c6accded1b14cf4de6, user_id=a0449380d9894626b5ead93cf60e96da, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:57.155617 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7ee4a0-6add-4b08-a21a-69ffcad19290 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:57.208455 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 976/1950] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:14:57 2022] PUT /identity/v3/projects/7d7da3c84e0746c6accded1b14cf4de6/users/a0449380d9894626b5ead93cf60e96da/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:14:57.221260 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f298209-d9a1-460f-b222-4d6d0854f9ed None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:57.222008 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f298209-d9a1-460f-b222-4d6d0854f9ed None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:57.222269 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f298209-d9a1-460f-b222-4d6d0854f9ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:57.323891 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-0f298209-d9a1-460f-b222-4d6d0854f9ed None None] MFA Rules not processed for user `a0449380d9894626b5ead93cf60e96da`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:14:57.326984 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-0f298209-d9a1-460f-b222-4d6d0854f9ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:57.569234 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 975/1951] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:14:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:14:57.574346 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7ee4a0-6add-4b08-a21a-69ffcad19290 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:57.723730 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7ee4a0-6add-4b08-a21a-69ffcad19290 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:57.731874 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a542a0-cc0b-434c-b283-2457b776f103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:57.733323 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a542a0-cc0b-434c-b283-2457b776f103 admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:57.733722 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a542a0-cc0b-434c-b283-2457b776f103 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:57.734081 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a542a0-cc0b-434c-b283-2457b776f103 admin admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:57.735381 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a542a0-cc0b-434c-b283-2457b776f103 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:57.739369 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a542a0-cc0b-434c-b283-2457b776f103 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:57.801850 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 977/1952] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:14:57 2022] POST /identity/v3/projects => generated 375 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:14:57.807550 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f298209-d9a1-460f-b222-4d6d0854f9ed None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:57.826254 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f298209-d9a1-460f-b222-4d6d0854f9ed None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:57.965946 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3920f7-5738-4f78-b97f-bbf8f3f4422f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:57.967291 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3920f7-5738-4f78-b97f-bbf8f3f4422f admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:57.968009 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3920f7-5738-4f78-b97f-bbf8f3f4422f admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:57.968661 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3920f7-5738-4f78-b97f-bbf8f3f4422f admin admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:57.969987 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3920f7-5738-4f78-b97f-bbf8f3f4422f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:57.974031 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3920f7-5738-4f78-b97f-bbf8f3f4422f admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:57.992533 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:58.016718 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a542a0-cc0b-434c-b283-2457b776f103 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:58.026558 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 976/1953] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:14:57 2022] POST /identity/v3/users => generated 288 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:14:58.033627 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a542a0-cc0b-434c-b283-2457b776f103 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:58.040791 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f134c17-6454-4a06-9c82-33c932f29046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:58.041563 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f134c17-6454-4a06-9c82-33c932f29046 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:58.042113 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f134c17-6454-4a06-9c82-33c932f29046 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:58.044140 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f134c17-6454-4a06-9c82-33c932f29046 admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:58.044140 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f134c17-6454-4a06-9c82-33c932f29046 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:58.046054 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f134c17-6454-4a06-9c82-33c932f29046 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:58.070344 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 978/1954] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:14:58 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 16:14:58.072081 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3920f7-5738-4f78-b97f-bbf8f3f4422f admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:58.086170 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3920f7-5738-4f78-b97f-bbf8f3f4422f admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:58.093711 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c95bdbc-0d91-40ce-9004-7b77c8d16f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:58.094592 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c95bdbc-0d91-40ce-9004-7b77c8d16f82 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:58.096031 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c95bdbc-0d91-40ce-9004-7b77c8d16f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:58.096384 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c95bdbc-0d91-40ce-9004-7b77c8d16f82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:58.097167 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c95bdbc-0d91-40ce-9004-7b77c8d16f82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:58.099827 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c95bdbc-0d91-40ce-9004-7b77c8d16f82 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:58.120559 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 977/1955] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:14:58 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 16:14:58.121477 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f134c17-6454-4a06-9c82-33c932f29046 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:58.137627 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f134c17-6454-4a06-9c82-33c932f29046 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:58.147965 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404e433d-024b-4ef5-93c8-85a382216baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:58.150388 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404e433d-024b-4ef5-93c8-85a382216baf admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:58.151551 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404e433d-024b-4ef5-93c8-85a382216baf admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:58.152253 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404e433d-024b-4ef5-93c8-85a382216baf admin admin] PATH_INFO: `/v3/projects/01b748c5ddd74e71a3d25dabef086305/users/f96f4f8f3e1a44098229dc34f6ec7999/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:58.179363 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404e433d-024b-4ef5-93c8-85a382216baf admin admin] RBAC: Authorizing `identity:create_grant(project_id=01b748c5ddd74e71a3d25dabef086305, user_id=f96f4f8f3e1a44098229dc34f6ec7999, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:58.185397 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404e433d-024b-4ef5-93c8-85a382216baf admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:58.197857 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=89,keystone:DELETE=6,keystone:INSERT=5 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:58.221863 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 979/1956] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:14:58 2022] PUT /identity/v3/projects/01b748c5ddd74e71a3d25dabef086305/users/f96f4f8f3e1a44098229dc34f6ec7999/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:14:58.222711 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6f2301-4740-4961-8fe7-0f2373388dbe None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:58.223708 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6f2301-4740-4961-8fe7-0f2373388dbe None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:58.224091 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6f2301-4740-4961-8fe7-0f2373388dbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:58.268031 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-4f6f2301-4740-4961-8fe7-0f2373388dbe None None] MFA Rules not processed for user `f96f4f8f3e1a44098229dc34f6ec7999`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:14:58.270475 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-4f6f2301-4740-4961-8fe7-0f2373388dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:58.443062 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 978/1957] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:14:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:14:58.448706 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404e433d-024b-4ef5-93c8-85a382216baf admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:58.575333 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404e433d-024b-4ef5-93c8-85a382216baf admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:58.590186 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf88889-b18b-4c5f-8e09-9b47110b1c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:58.591013 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf88889-b18b-4c5f-8e09-9b47110b1c6e admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:58.591435 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf88889-b18b-4c5f-8e09-9b47110b1c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:58.591909 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf88889-b18b-4c5f-8e09-9b47110b1c6e admin admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:58.592725 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf88889-b18b-4c5f-8e09-9b47110b1c6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:58.600905 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf88889-b18b-4c5f-8e09-9b47110b1c6e admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:58.667661 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 980/1958] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:14:58 2022] POST /identity/v3/projects => generated 375 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:14:58.669719 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6f2301-4740-4961-8fe7-0f2373388dbe None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:58.704636 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6f2301-4740-4961-8fe7-0f2373388dbe None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:58.922312 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e548238-4f9f-4207-94fd-da40c62a7489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:58.924863 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e548238-4f9f-4207-94fd-da40c62a7489 admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:58.925503 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e548238-4f9f-4207-94fd-da40c62a7489 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:58.926071 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e548238-4f9f-4207-94fd-da40c62a7489 admin admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:58.928228 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e548238-4f9f-4207-94fd-da40c62a7489 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:58.935377 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e548238-4f9f-4207-94fd-da40c62a7489 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:59.013224 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf88889-b18b-4c5f-8e09-9b47110b1c6e admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:59.014181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 979/1959] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:14:58 2022] POST /identity/v3/users => generated 294 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:14:59.028573 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf88889-b18b-4c5f-8e09-9b47110b1c6e admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:59.057314 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55389ce-143e-4b9b-994e-2832bdde5435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:59.057978 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55389ce-143e-4b9b-994e-2832bdde5435 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:59.058279 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55389ce-143e-4b9b-994e-2832bdde5435 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:59.058615 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55389ce-143e-4b9b-994e-2832bdde5435 admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:59.059407 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55389ce-143e-4b9b-994e-2832bdde5435 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:59.064055 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55389ce-143e-4b9b-994e-2832bdde5435 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:59.095905 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 981/1960] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:14:59 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:14:59.097813 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e548238-4f9f-4207-94fd-da40c62a7489 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:59.135374 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e548238-4f9f-4207-94fd-da40c62a7489 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:59.160190 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4be0c0c-0d46-451f-ac9a-b0d09d8a8382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:59.161497 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4be0c0c-0d46-451f-ac9a-b0d09d8a8382 admin admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:59.162154 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4be0c0c-0d46-451f-ac9a-b0d09d8a8382 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:59.162847 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4be0c0c-0d46-451f-ac9a-b0d09d8a8382 admin admin] PATH_INFO: `/v3/projects/dfb3afb3441f4676b201ab93c9dfdefe/users/ca42b57d34f243dbaa2f916077987d7e/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:59.220416 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4be0c0c-0d46-451f-ac9a-b0d09d8a8382 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfb3afb3441f4676b201ab93c9dfdefe, user_id=ca42b57d34f243dbaa2f916077987d7e, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:59.227316 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4be0c0c-0d46-451f-ac9a-b0d09d8a8382 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:59.252685 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 980/1961] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:14:59 2022] PUT /identity/v3/projects/dfb3afb3441f4676b201ab93c9dfdefe/users/ca42b57d34f243dbaa2f916077987d7e/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:14:59.259029 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55389ce-143e-4b9b-994e-2832bdde5435 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:59.287334 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55389ce-143e-4b9b-994e-2832bdde5435 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:59.378203 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba935b2-5a2c-49ec-a660-4fab9974e5aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:59.379230 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba935b2-5a2c-49ec-a660-4fab9974e5aa admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:59.379656 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba935b2-5a2c-49ec-a660-4fab9974e5aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:59.380008 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba935b2-5a2c-49ec-a660-4fab9974e5aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:59.381395 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba935b2-5a2c-49ec-a660-4fab9974e5aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:59.383310 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba935b2-5a2c-49ec-a660-4fab9974e5aa admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:59.397283 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 982/1962] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:14:59 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:14:59.402216 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4be0c0c-0d46-451f-ac9a-b0d09d8a8382 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:59.414254 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4be0c0c-0d46-451f-ac9a-b0d09d8a8382 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:59.422056 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c401be5a-70af-4c60-8e55-7e26cbc626c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:59.423429 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c401be5a-70af-4c60-8e55-7e26cbc626c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:59.423889 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c401be5a-70af-4c60-8e55-7e26cbc626c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:59.424356 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c401be5a-70af-4c60-8e55-7e26cbc626c1 admin admin] PATH_INFO: `/v3/projects/dfb3afb3441f4676b201ab93c9dfdefe/users/ca42b57d34f243dbaa2f916077987d7e/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:59.428188 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c401be5a-70af-4c60-8e55-7e26cbc626c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfb3afb3441f4676b201ab93c9dfdefe, user_id=ca42b57d34f243dbaa2f916077987d7e, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:59.432799 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c401be5a-70af-4c60-8e55-7e26cbc626c1 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:59.450263 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 981/1963] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:14:59 2022] PUT /identity/v3/projects/dfb3afb3441f4676b201ab93c9dfdefe/users/ca42b57d34f243dbaa2f916077987d7e/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:14:59.467296 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d169ce56-4ecd-433e-b38e-8cb67e8045bb None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:59.467641 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d169ce56-4ecd-433e-b38e-8cb67e8045bb None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:59.467897 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d169ce56-4ecd-433e-b38e-8cb67e8045bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:59.504492 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-d169ce56-4ecd-433e-b38e-8cb67e8045bb None None] MFA Rules not processed for user `ca42b57d34f243dbaa2f916077987d7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:14:59.505737 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-d169ce56-4ecd-433e-b38e-8cb67e8045bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:59.604852 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:59.624031 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 983/1964] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:14:59 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:14:59.637352 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c401be5a-70af-4c60-8e55-7e26cbc626c1 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:14:59.640250 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-c401be5a-70af-4c60-8e55-7e26cbc626c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:14:59.680778 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:59.720540 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c401be5a-70af-4c60-8e55-7e26cbc626c1 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:14:59.725242 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5740f54-fdea-4222-b2ea-25d500a6cf15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:14:59.726050 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5740f54-fdea-4222-b2ea-25d500a6cf15 service cinder] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:14:59.726409 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5740f54-fdea-4222-b2ea-25d500a6cf15 service cinder] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:14:59.726763 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5740f54-fdea-4222-b2ea-25d500a6cf15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:14:59.863212 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5740f54-fdea-4222-b2ea-25d500a6cf15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:14:59.869447 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5740f54-fdea-4222-b2ea-25d500a6cf15 service cinder] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:14:59.896761 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 982/1965] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:14:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:15:01.884814 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d169ce56-4ecd-433e-b38e-8cb67e8045bb None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:01.907015 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d169ce56-4ecd-433e-b38e-8cb67e8045bb None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:01.916671 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c599827b-fcb4-403a-9eea-a7df9239faf1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:01.923882 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c599827b-fcb4-403a-9eea-a7df9239faf1 service cinder] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:01.924743 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c599827b-fcb4-403a-9eea-a7df9239faf1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:01.925398 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c599827b-fcb4-403a-9eea-a7df9239faf1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:02.057668 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c599827b-fcb4-403a-9eea-a7df9239faf1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:02.061142 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c599827b-fcb4-403a-9eea-a7df9239faf1 service cinder] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:02.096315 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 984/1966] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:15:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:15:02.826576 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5740f54-fdea-4222-b2ea-25d500a6cf15 service cinder] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:02.842282 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5740f54-fdea-4222-b2ea-25d500a6cf15 service cinder] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:02.848535 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30abb81-ca56-45bc-8278-045604506d1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:02.850210 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30abb81-ca56-45bc-8278-045604506d1e service cinder] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:02.850716 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30abb81-ca56-45bc-8278-045604506d1e service cinder] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:02.851168 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30abb81-ca56-45bc-8278-045604506d1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:02.869015 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30abb81-ca56-45bc-8278-045604506d1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:02.871946 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30abb81-ca56-45bc-8278-045604506d1e service cinder] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:02.891017 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 983/1967] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:15:02 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:15:06.211781 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144f2318-6fc8-46b1-93cb-c62f6c8de0a4 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:06.212747 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144f2318-6fc8-46b1-93cb-c62f6c8de0a4 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:06.213171 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144f2318-6fc8-46b1-93cb-c62f6c8de0a4 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:06.215236 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 985/1968] 10.222.0.57 () {54 vars in 921 bytes} [Mon Sep 19 16:15:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:15:06.246604 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c94b3c-8f49-4ffa-bbce-f7754ce90fee None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:06.247149 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c94b3c-8f49-4ffa-bbce-f7754ce90fee None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:06.247383 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c94b3c-8f49-4ffa-bbce-f7754ce90fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:06.283270 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-34c94b3c-8f49-4ffa-bbce-f7754ce90fee None None] MFA Rules not processed for user `1f1a985495ed48a4acc48e2e6fc7f977`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:15:06.284536 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-34c94b3c-8f49-4ffa-bbce-f7754ce90fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:15:06.413620 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 984/1969] 10.222.0.57 () {58 vars in 1018 bytes} [Mon Sep 19 16:15:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:15:06.456982 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144f2318-6fc8-46b1-93cb-c62f6c8de0a4 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:06.461119 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-144f2318-6fc8-46b1-93cb-c62f6c8de0a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:15:06.534889 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144f2318-6fc8-46b1-93cb-c62f6c8de0a4 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:06.552849 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acbde7b-de8b-4ccd-ac31-9661a3342e7e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1a985495ed48a4acc48e2e6fc7f977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:06.553906 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acbde7b-de8b-4ccd-ac31-9661a3342e7e service placement] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:06.554452 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acbde7b-de8b-4ccd-ac31-9661a3342e7e service placement] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:06.555060 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acbde7b-de8b-4ccd-ac31-9661a3342e7e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:06.582022 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acbde7b-de8b-4ccd-ac31-9661a3342e7e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:06.587102 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acbde7b-de8b-4ccd-ac31-9661a3342e7e service placement] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:06.617290 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 986/1970] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:15:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:15:07.379234 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c94b3c-8f49-4ffa-bbce-f7754ce90fee None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:07.490478 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c94b3c-8f49-4ffa-bbce-f7754ce90fee None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:07.499490 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdba29b-9757-447b-8f85-786f2dae194c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:07.500635 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdba29b-9757-447b-8f85-786f2dae194c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:07.500913 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdba29b-9757-447b-8f85-786f2dae194c admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:07.501345 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdba29b-9757-447b-8f85-786f2dae194c admin admin] PATH_INFO: `/v3/users/a0449380d9894626b5ead93cf60e96da` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:07.503014 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdba29b-9757-447b-8f85-786f2dae194c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0449380d9894626b5ead93cf60e96da)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:07.507352 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdba29b-9757-447b-8f85-786f2dae194c admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:07.517213 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:15:07.596145 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-8cdba29b-9757-447b-8f85-786f2dae194c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0449380d9894626b5ead93cf60e96da'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:07.597571 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-8cdba29b-9757-447b-8f85-786f2dae194c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0449380d9894626b5ead93cf60e96da'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:07.609814 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-8cdba29b-9757-447b-8f85-786f2dae194c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0449380d9894626b5ead93cf60e96da'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:07.635043 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-8cdba29b-9757-447b-8f85-786f2dae194c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0449380d9894626b5ead93cf60e96da'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:07.636848 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-8cdba29b-9757-447b-8f85-786f2dae194c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0449380d9894626b5ead93cf60e96da'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:07.662932 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cc9f86-ecb6-45cc-994b-2256a23451d2 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:07.664539 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cc9f86-ecb6-45cc-994b-2256a23451d2 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:07.665284 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cc9f86-ecb6-45cc-994b-2256a23451d2 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:07.670474 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 985/1971] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:15:07 2022] DELETE /identity/v3/users/a0449380d9894626b5ead93cf60e96da => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:15:07.671866 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 987/1972] 10.222.0.57 () {54 vars in 921 bytes} [Mon Sep 19 16:15:07 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:15:07.673636 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf0cba5-a085-4854-a345-e363b974d553 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:07.674141 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf0cba5-a085-4854-a345-e363b974d553 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:07.674792 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf0cba5-a085-4854-a345-e363b974d553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:07.719590 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-daf0cba5-a085-4854-a345-e363b974d553 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:15:07.721578 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-daf0cba5-a085-4854-a345-e363b974d553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:15:07.728009 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d1aafd-3079-4da9-8f5e-482e32e2f4c4 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:07.728683 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d1aafd-3079-4da9-8f5e-482e32e2f4c4 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:07.729130 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d1aafd-3079-4da9-8f5e-482e32e2f4c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:07.769599 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-93d1aafd-3079-4da9-8f5e-482e32e2f4c4 None None] MFA Rules not processed for user `1f1a985495ed48a4acc48e2e6fc7f977`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:15:07.772365 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-93d1aafd-3079-4da9-8f5e-482e32e2f4c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:15:07.856024 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 986/1973] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:15:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:15:07.856540 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf0cba5-a085-4854-a345-e363b974d553 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:07.860287 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-daf0cba5-a085-4854-a345-e363b974d553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:15:07.874284 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf0cba5-a085-4854-a345-e363b974d553 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:07.881354 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52471303-3eb8-4471-861c-7cc21cdf808f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:07.882180 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52471303-3eb8-4471-861c-7cc21cdf808f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:07.882521 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52471303-3eb8-4471-861c-7cc21cdf808f admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:07.882882 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52471303-3eb8-4471-861c-7cc21cdf808f admin admin] PATH_INFO: `/v3/users/04ec95dde7c245e598d6ca82f476eb49` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:07.884141 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52471303-3eb8-4471-861c-7cc21cdf808f admin admin] RBAC: Authorizing `identity:delete_user(user_id=04ec95dde7c245e598d6ca82f476eb49)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:07.886334 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52471303-3eb8-4471-861c-7cc21cdf808f admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:07.904462 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 988/1974] 10.222.0.57 () {58 vars in 1018 bytes} [Mon Sep 19 16:15:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:15:07.905657 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d1aafd-3079-4da9-8f5e-482e32e2f4c4 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:07.907530 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-93d1aafd-3079-4da9-8f5e-482e32e2f4c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:15:07.949832 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-52471303-3eb8-4471-861c-7cc21cdf808f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04ec95dde7c245e598d6ca82f476eb49'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:07.951033 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-52471303-3eb8-4471-861c-7cc21cdf808f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04ec95dde7c245e598d6ca82f476eb49'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:07.964624 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-52471303-3eb8-4471-861c-7cc21cdf808f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04ec95dde7c245e598d6ca82f476eb49'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:07.979486 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-52471303-3eb8-4471-861c-7cc21cdf808f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04ec95dde7c245e598d6ca82f476eb49'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:07.980261 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-52471303-3eb8-4471-861c-7cc21cdf808f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04ec95dde7c245e598d6ca82f476eb49'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:07.984755 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:15:07.991239 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d1aafd-3079-4da9-8f5e-482e32e2f4c4 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:07.999948 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73e05c5-754a-4324-93de-387fe51faaf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:08.000911 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 987/1975] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:15:07 2022] DELETE /identity/v3/users/04ec95dde7c245e598d6ca82f476eb49 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:15:08.001300 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e05c5-754a-4324-93de-387fe51faaf5 service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:08.001841 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e05c5-754a-4324-93de-387fe51faaf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:08.002325 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e05c5-754a-4324-93de-387fe51faaf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:08.005286 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52471303-3eb8-4471-861c-7cc21cdf808f admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:08.006645 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-52471303-3eb8-4471-861c-7cc21cdf808f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:15:08.010475 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-a73e05c5-754a-4324-93de-387fe51faaf5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:15:08.033386 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73e05c5-754a-4324-93de-387fe51faaf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:08.036150 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73e05c5-754a-4324-93de-387fe51faaf5 service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:08.056125 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 989/1976] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:15:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:15:08.150186 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52471303-3eb8-4471-861c-7cc21cdf808f admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:08.161218 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94c986d-f894-4302-b6d5-38e14fbd9d78 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1a985495ed48a4acc48e2e6fc7f977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:08.163027 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c986d-f894-4302-b6d5-38e14fbd9d78 service placement] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:08.163583 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c986d-f894-4302-b6d5-38e14fbd9d78 service placement] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:08.164074 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c986d-f894-4302-b6d5-38e14fbd9d78 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:08.167803 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-f94c986d-f894-4302-b6d5-38e14fbd9d78 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:15:08.193807 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94c986d-f894-4302-b6d5-38e14fbd9d78 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:08.197451 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94c986d-f894-4302-b6d5-38e14fbd9d78 service placement] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:08.229673 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 988/1977] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:15:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:15:08.351306 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73e05c5-754a-4324-93de-387fe51faaf5 service neutron] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:08.353572 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-a73e05c5-754a-4324-93de-387fe51faaf5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:15:08.375415 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73e05c5-754a-4324-93de-387fe51faaf5 service neutron] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:08.403074 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:15:08.494467 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa50ab2d-9d3d-4b67-8248-80adee47989d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:08.495232 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa50ab2d-9d3d-4b67-8248-80adee47989d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:08.495613 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa50ab2d-9d3d-4b67-8248-80adee47989d admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:08.496250 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa50ab2d-9d3d-4b67-8248-80adee47989d admin admin] PATH_INFO: `/v3/projects/15abe68f64cd487590d31fb8eabd28ae` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:08.498272 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa50ab2d-9d3d-4b67-8248-80adee47989d admin admin] RBAC: Authorizing `identity:delete_project(project_id=15abe68f64cd487590d31fb8eabd28ae)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:08.500455 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa50ab2d-9d3d-4b67-8248-80adee47989d admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:08.576200 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-aa50ab2d-9d3d-4b67-8248-80adee47989d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15abe68f64cd487590d31fb8eabd28ae', 'request_id': 'req-aa50ab2d-9d3d-4b67-8248-80adee47989d'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:08.584680 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-aa50ab2d-9d3d-4b67-8248-80adee47989d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15abe68f64cd487590d31fb8eabd28ae', 'request_id': 'req-aa50ab2d-9d3d-4b67-8248-80adee47989d'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:08.595083 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-aa50ab2d-9d3d-4b67-8248-80adee47989d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15abe68f64cd487590d31fb8eabd28ae', 'request_id': 'req-aa50ab2d-9d3d-4b67-8248-80adee47989d'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:08.616469 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-aa50ab2d-9d3d-4b67-8248-80adee47989d admin admin] The token cache is being invalidate because project 15abe68f64cd487590d31fb8eabd28ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81618) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:15:08.616773 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-aa50ab2d-9d3d-4b67-8248-80adee47989d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:08.617598 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-aa50ab2d-9d3d-4b67-8248-80adee47989d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:08.619198 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 990/1978] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:15:08 2022] DELETE /identity/v3/projects/15abe68f64cd487590d31fb8eabd28ae => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:15:08.623831 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94c986d-f894-4302-b6d5-38e14fbd9d78 service placement] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:08.637527 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94c986d-f894-4302-b6d5-38e14fbd9d78 service placement] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:08.646048 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa50ab2d-9d3d-4b67-8248-80adee47989d admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:08.648507 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-aa50ab2d-9d3d-4b67-8248-80adee47989d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:15:08.783464 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa50ab2d-9d3d-4b67-8248-80adee47989d admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:08.792027 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473fe616-7143-4fc7-9bfd-4348142ca7a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:08.793175 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473fe616-7143-4fc7-9bfd-4348142ca7a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:08.793729 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473fe616-7143-4fc7-9bfd-4348142ca7a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:08.794273 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473fe616-7143-4fc7-9bfd-4348142ca7a4 admin admin] PATH_INFO: `/v3/projects/7d7da3c84e0746c6accded1b14cf4de6` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:08.796574 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473fe616-7143-4fc7-9bfd-4348142ca7a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d7da3c84e0746c6accded1b14cf4de6)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:08.799301 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c64e23-edfc-44a0-9bc4-ea9ddc867be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:08.800204 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473fe616-7143-4fc7-9bfd-4348142ca7a4 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:08.801004 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c64e23-edfc-44a0-9bc4-ea9ddc867be5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:08.801708 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c64e23-edfc-44a0-9bc4-ea9ddc867be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:08.804425 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c64e23-edfc-44a0-9bc4-ea9ddc867be5 admin admin] PATH_INFO: `/v3/users/218615224abf468c85c11a3d97914ee4` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:08.806950 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c64e23-edfc-44a0-9bc4-ea9ddc867be5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=218615224abf468c85c11a3d97914ee4)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:08.809551 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c64e23-edfc-44a0-9bc4-ea9ddc867be5 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:08.924969 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-13c64e23-edfc-44a0-9bc4-ea9ddc867be5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '218615224abf468c85c11a3d97914ee4'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:08.926284 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-13c64e23-edfc-44a0-9bc4-ea9ddc867be5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '218615224abf468c85c11a3d97914ee4'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:08.956215 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-13c64e23-edfc-44a0-9bc4-ea9ddc867be5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '218615224abf468c85c11a3d97914ee4'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:08.956969 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-473fe616-7143-4fc7-9bfd-4348142ca7a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d7da3c84e0746c6accded1b14cf4de6', 'request_id': 'req-473fe616-7143-4fc7-9bfd-4348142ca7a4'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:08.963400 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:15:08.987138 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-473fe616-7143-4fc7-9bfd-4348142ca7a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d7da3c84e0746c6accded1b14cf4de6', 'request_id': 'req-473fe616-7143-4fc7-9bfd-4348142ca7a4'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:08.993388 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-13c64e23-edfc-44a0-9bc4-ea9ddc867be5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '218615224abf468c85c11a3d97914ee4'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:08.994585 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-13c64e23-edfc-44a0-9bc4-ea9ddc867be5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '218615224abf468c85c11a3d97914ee4'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:09.006087 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-473fe616-7143-4fc7-9bfd-4348142ca7a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d7da3c84e0746c6accded1b14cf4de6', 'request_id': 'req-473fe616-7143-4fc7-9bfd-4348142ca7a4'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:09.020009 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 989/1979] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:15:08 2022] DELETE /identity/v3/users/218615224abf468c85c11a3d97914ee4 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 16:15:09.047427 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-473fe616-7143-4fc7-9bfd-4348142ca7a4 admin admin] The token cache is being invalidate because project 7d7da3c84e0746c6accded1b14cf4de6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81618) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:15:09.048417 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-473fe616-7143-4fc7-9bfd-4348142ca7a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:09.051407 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-473fe616-7143-4fc7-9bfd-4348142ca7a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:09.056180 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 991/1980] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:15:08 2022] DELETE /identity/v3/projects/7d7da3c84e0746c6accded1b14cf4de6 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 16:15:09.060970 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c64e23-edfc-44a0-9bc4-ea9ddc867be5 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:09.063684 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-13c64e23-edfc-44a0-9bc4-ea9ddc867be5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:15:09.073646 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:15:09.175448 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c64e23-edfc-44a0-9bc4-ea9ddc867be5 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:09.182090 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aefd71-5aeb-4144-b407-97ef11d59bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:09.183067 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aefd71-5aeb-4144-b407-97ef11d59bfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:09.184014 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aefd71-5aeb-4144-b407-97ef11d59bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:09.184014 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aefd71-5aeb-4144-b407-97ef11d59bfa admin admin] PATH_INFO: `/v3/users/f96f4f8f3e1a44098229dc34f6ec7999` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:09.185752 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aefd71-5aeb-4144-b407-97ef11d59bfa admin admin] RBAC: Authorizing `identity:delete_user(user_id=f96f4f8f3e1a44098229dc34f6ec7999)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:09.188330 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aefd71-5aeb-4144-b407-97ef11d59bfa admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:09.263113 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-25aefd71-5aeb-4144-b407-97ef11d59bfa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f96f4f8f3e1a44098229dc34f6ec7999'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:09.264163 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-25aefd71-5aeb-4144-b407-97ef11d59bfa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f96f4f8f3e1a44098229dc34f6ec7999'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:09.278103 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-25aefd71-5aeb-4144-b407-97ef11d59bfa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f96f4f8f3e1a44098229dc34f6ec7999'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:09.313062 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-25aefd71-5aeb-4144-b407-97ef11d59bfa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f96f4f8f3e1a44098229dc34f6ec7999'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:09.314662 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-25aefd71-5aeb-4144-b407-97ef11d59bfa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f96f4f8f3e1a44098229dc34f6ec7999'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:09.342174 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 990/1981] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:15:09 2022] DELETE /identity/v3/users/f96f4f8f3e1a44098229dc34f6ec7999 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:15:09.405052 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473fe616-7143-4fc7-9bfd-4348142ca7a4 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:09.408112 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-473fe616-7143-4fc7-9bfd-4348142ca7a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:15:09.578998 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473fe616-7143-4fc7-9bfd-4348142ca7a4 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:09.724281 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84048379-b6a2-4d51-94ad-50c4c9a1bcd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:09.725492 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84048379-b6a2-4d51-94ad-50c4c9a1bcd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:09.725987 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84048379-b6a2-4d51-94ad-50c4c9a1bcd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:09.726534 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84048379-b6a2-4d51-94ad-50c4c9a1bcd1 admin admin] PATH_INFO: `/v3/projects/b4a885a9387e4d50a1ad3ef26419a909` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:09.729415 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84048379-b6a2-4d51-94ad-50c4c9a1bcd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4a885a9387e4d50a1ad3ef26419a909)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:09.731853 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84048379-b6a2-4d51-94ad-50c4c9a1bcd1 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:09.819605 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-84048379-b6a2-4d51-94ad-50c4c9a1bcd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4a885a9387e4d50a1ad3ef26419a909', 'request_id': 'req-84048379-b6a2-4d51-94ad-50c4c9a1bcd1'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:09.828751 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-84048379-b6a2-4d51-94ad-50c4c9a1bcd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4a885a9387e4d50a1ad3ef26419a909', 'request_id': 'req-84048379-b6a2-4d51-94ad-50c4c9a1bcd1'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:09.840672 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-84048379-b6a2-4d51-94ad-50c4c9a1bcd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4a885a9387e4d50a1ad3ef26419a909', 'request_id': 'req-84048379-b6a2-4d51-94ad-50c4c9a1bcd1'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:09.863797 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-84048379-b6a2-4d51-94ad-50c4c9a1bcd1 admin admin] The token cache is being invalidate because project b4a885a9387e4d50a1ad3ef26419a909 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81618) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:15:09.864308 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-84048379-b6a2-4d51-94ad-50c4c9a1bcd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:09.865987 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-84048379-b6a2-4d51-94ad-50c4c9a1bcd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:09.869037 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 992/1982] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:15:09 2022] DELETE /identity/v3/projects/b4a885a9387e4d50a1ad3ef26419a909 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:15:09.874406 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c900b-662e-4b16-aa87-bde109bbd420 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:09.875027 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c900b-662e-4b16-aa87-bde109bbd420 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:09.875027 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c900b-662e-4b16-aa87-bde109bbd420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:09.899215 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-050c900b-662e-4b16-aa87-bde109bbd420 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:15:09.900601 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-050c900b-662e-4b16-aa87-bde109bbd420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:15:10.019083 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 991/1983] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:15:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:15:10.029830 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c127b27a-967f-44e1-9e4a-5ccf06291f34 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:10.030346 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c127b27a-967f-44e1-9e4a-5ccf06291f34 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:10.030889 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c127b27a-967f-44e1-9e4a-5ccf06291f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:10.068441 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-c127b27a-967f-44e1-9e4a-5ccf06291f34 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:15:10.072170 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-c127b27a-967f-44e1-9e4a-5ccf06291f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:15:10.091591 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 993/1984] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:15:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:15:10.097628 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050c900b-662e-4b16-aa87-bde109bbd420 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:10.101513 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c127b27a-967f-44e1-9e4a-5ccf06291f34 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:10.104448 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-c127b27a-967f-44e1-9e4a-5ccf06291f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:15:10.125100 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c127b27a-967f-44e1-9e4a-5ccf06291f34 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:10.131353 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb353506-79bf-4a4b-ac41-196af5c160f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:10.132004 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb353506-79bf-4a4b-ac41-196af5c160f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:10.132350 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb353506-79bf-4a4b-ac41-196af5c160f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:10.133463 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb353506-79bf-4a4b-ac41-196af5c160f7 admin admin] PATH_INFO: `/v3/projects/01b748c5ddd74e71a3d25dabef086305` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:10.133948 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb353506-79bf-4a4b-ac41-196af5c160f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01b748c5ddd74e71a3d25dabef086305)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:10.135977 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb353506-79bf-4a4b-ac41-196af5c160f7 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:10.140688 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050c900b-662e-4b16-aa87-bde109bbd420 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:10.148411 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:15:10.181801 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e159ae22-829b-4862-a52c-8f22735f0feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:10.182981 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159ae22-829b-4862-a52c-8f22735f0feb admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:10.183461 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159ae22-829b-4862-a52c-8f22735f0feb admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:10.183979 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159ae22-829b-4862-a52c-8f22735f0feb admin admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:10.185373 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e159ae22-829b-4862-a52c-8f22735f0feb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:15:10.186005 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e159ae22-829b-4862-a52c-8f22735f0feb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:10.191162 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e159ae22-829b-4862-a52c-8f22735f0feb admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:10.236449 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 992/1985] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:15:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:15:10.240240 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e159ae22-829b-4862-a52c-8f22735f0feb admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:10.258867 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-fb353506-79bf-4a4b-ac41-196af5c160f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01b748c5ddd74e71a3d25dabef086305', 'request_id': 'req-fb353506-79bf-4a4b-ac41-196af5c160f7'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:10.261054 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e159ae22-829b-4862-a52c-8f22735f0feb admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:10.275353 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-fb353506-79bf-4a4b-ac41-196af5c160f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01b748c5ddd74e71a3d25dabef086305', 'request_id': 'req-fb353506-79bf-4a4b-ac41-196af5c160f7'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:10.277065 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:15:10.287505 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-fb353506-79bf-4a4b-ac41-196af5c160f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01b748c5ddd74e71a3d25dabef086305', 'request_id': 'req-fb353506-79bf-4a4b-ac41-196af5c160f7'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:10.327530 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-fb353506-79bf-4a4b-ac41-196af5c160f7 admin admin] The token cache is being invalidate because project 01b748c5ddd74e71a3d25dabef086305 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81618) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:15:10.328247 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-fb353506-79bf-4a4b-ac41-196af5c160f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:10.329909 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-fb353506-79bf-4a4b-ac41-196af5c160f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:10.333737 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 994/1986] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:15:10 2022] DELETE /identity/v3/projects/01b748c5ddd74e71a3d25dabef086305 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:15:10.339348 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb353506-79bf-4a4b-ac41-196af5c160f7 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:10.343345 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-fb353506-79bf-4a4b-ac41-196af5c160f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:15:10.398105 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0745a7e4-d44f-44fc-b1ca-4605a22d87e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:10.399318 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0745a7e4-d44f-44fc-b1ca-4605a22d87e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:10.399926 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0745a7e4-d44f-44fc-b1ca-4605a22d87e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:10.400395 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0745a7e4-d44f-44fc-b1ca-4605a22d87e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:10.401573 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0745a7e4-d44f-44fc-b1ca-4605a22d87e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:10.405646 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0745a7e4-d44f-44fc-b1ca-4605a22d87e6 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:10.439425 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb353506-79bf-4a4b-ac41-196af5c160f7 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:10.445524 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640fb03b-070b-4d07-95d0-8c36c225d210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:10.446480 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640fb03b-070b-4d07-95d0-8c36c225d210 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:10.446860 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640fb03b-070b-4d07-95d0-8c36c225d210 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:10.447226 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640fb03b-070b-4d07-95d0-8c36c225d210 admin admin] PATH_INFO: `/v3/users/ca42b57d34f243dbaa2f916077987d7e` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:10.448906 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640fb03b-070b-4d07-95d0-8c36c225d210 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca42b57d34f243dbaa2f916077987d7e)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:10.450761 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640fb03b-070b-4d07-95d0-8c36c225d210 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:10.468156 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 993/1987] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:15:10 2022] POST /identity/v3/projects => generated 375 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:15:10.473881 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0745a7e4-d44f-44fc-b1ca-4605a22d87e6 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:10.478055 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-0745a7e4-d44f-44fc-b1ca-4605a22d87e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:15:10.547811 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-640fb03b-070b-4d07-95d0-8c36c225d210 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca42b57d34f243dbaa2f916077987d7e'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:10.557227 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-640fb03b-070b-4d07-95d0-8c36c225d210 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca42b57d34f243dbaa2f916077987d7e'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:10.577907 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-640fb03b-070b-4d07-95d0-8c36c225d210 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca42b57d34f243dbaa2f916077987d7e'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:10.591958 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-640fb03b-070b-4d07-95d0-8c36c225d210 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca42b57d34f243dbaa2f916077987d7e'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:10.592753 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-640fb03b-070b-4d07-95d0-8c36c225d210 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca42b57d34f243dbaa2f916077987d7e'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:10.599922 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0745a7e4-d44f-44fc-b1ca-4605a22d87e6 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:10.601398 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 995/1988] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:15:10 2022] DELETE /identity/v3/users/ca42b57d34f243dbaa2f916077987d7e => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:15:10.708464 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0883ec4-72bc-430e-b235-a53999b4fca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:10.709095 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0883ec4-72bc-430e-b235-a53999b4fca2 admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:10.709377 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0883ec4-72bc-430e-b235-a53999b4fca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:10.709653 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0883ec4-72bc-430e-b235-a53999b4fca2 admin admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:10.710313 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0883ec4-72bc-430e-b235-a53999b4fca2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:10.712466 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0883ec4-72bc-430e-b235-a53999b4fca2 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:10.747609 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 994/1989] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:15:10 2022] POST /identity/v3/users => generated 288 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 16:15:10.751985 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640fb03b-070b-4d07-95d0-8c36c225d210 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:10.753533 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-640fb03b-070b-4d07-95d0-8c36c225d210 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:15:10.773484 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640fb03b-070b-4d07-95d0-8c36c225d210 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:10.779699 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8494d102-1c0b-4540-8eea-1fa838f7092d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:10.781190 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8494d102-1c0b-4540-8eea-1fa838f7092d admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:10.781874 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8494d102-1c0b-4540-8eea-1fa838f7092d admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:10.782350 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8494d102-1c0b-4540-8eea-1fa838f7092d admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:10.783592 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8494d102-1c0b-4540-8eea-1fa838f7092d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:10.786062 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8494d102-1c0b-4540-8eea-1fa838f7092d admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:10.797653 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:15:10.809801 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 996/1990] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:15:10 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:15:10.814053 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0883ec4-72bc-430e-b235-a53999b4fca2 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:10.834875 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0883ec4-72bc-430e-b235-a53999b4fca2 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:10.845815 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e1be74-2db0-4ec4-933d-9491aa142c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:10.847534 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1be74-2db0-4ec4-933d-9491aa142c46 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:10.847534 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1be74-2db0-4ec4-933d-9491aa142c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:10.847984 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1be74-2db0-4ec4-933d-9491aa142c46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:10.849738 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e1be74-2db0-4ec4-933d-9491aa142c46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:10.852779 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e1be74-2db0-4ec4-933d-9491aa142c46 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:10.873085 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 995/1991] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:15:10 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:15:10.879850 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8494d102-1c0b-4540-8eea-1fa838f7092d admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:10.899601 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8494d102-1c0b-4540-8eea-1fa838f7092d admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:10.909416 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb5f705-3325-4383-b7fc-3ae816886a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:10.910607 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb5f705-3325-4383-b7fc-3ae816886a73 admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:10.911038 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb5f705-3325-4383-b7fc-3ae816886a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:10.911979 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb5f705-3325-4383-b7fc-3ae816886a73 admin admin] PATH_INFO: `/v3/projects/35121c2a50974cbfab5bd219a1b164ec/users/5c397ef8d5b34358a6afeb85ea551006/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:10.947102 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb5f705-3325-4383-b7fc-3ae816886a73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35121c2a50974cbfab5bd219a1b164ec, user_id=5c397ef8d5b34358a6afeb85ea551006, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:10.953129 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb5f705-3325-4383-b7fc-3ae816886a73 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:10.977404 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 997/1992] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:15:10 2022] PUT /identity/v3/projects/35121c2a50974cbfab5bd219a1b164ec/users/5c397ef8d5b34358a6afeb85ea551006/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:15:10.997372 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c49f9fe-66f1-4012-9764-bb056bcdd17e None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:10.998303 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c49f9fe-66f1-4012-9764-bb056bcdd17e None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:10.998303 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c49f9fe-66f1-4012-9764-bb056bcdd17e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:11.042216 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-1c49f9fe-66f1-4012-9764-bb056bcdd17e None None] MFA Rules not processed for user `5c397ef8d5b34358a6afeb85ea551006`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:15:11.044355 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-1c49f9fe-66f1-4012-9764-bb056bcdd17e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:15:11.064818 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:15:11.111934 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb5f705-3325-4383-b7fc-3ae816886a73 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:11.114897 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-3cb5f705-3325-4383-b7fc-3ae816886a73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:15:11.202936 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 996/1993] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:15:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:15:11.231415 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c49f9fe-66f1-4012-9764-bb056bcdd17e None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:11.234144 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-1c49f9fe-66f1-4012-9764-bb056bcdd17e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:15:11.278572 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb5f705-3325-4383-b7fc-3ae816886a73 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:11.283831 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a314f8-bc6e-4ae5-b5e2-533007b38898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:11.284488 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a314f8-bc6e-4ae5-b5e2-533007b38898 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:11.284915 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a314f8-bc6e-4ae5-b5e2-533007b38898 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:11.285200 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a314f8-bc6e-4ae5-b5e2-533007b38898 admin admin] PATH_INFO: `/v3/projects/dfb3afb3441f4676b201ab93c9dfdefe` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:11.286548 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a314f8-bc6e-4ae5-b5e2-533007b38898 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfb3afb3441f4676b201ab93c9dfdefe)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:11.288603 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a314f8-bc6e-4ae5-b5e2-533007b38898 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:11.383607 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-e4a314f8-bc6e-4ae5-b5e2-533007b38898 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfb3afb3441f4676b201ab93c9dfdefe', 'request_id': 'req-e4a314f8-bc6e-4ae5-b5e2-533007b38898'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:11.385638 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c49f9fe-66f1-4012-9764-bb056bcdd17e None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:11.393534 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-e4a314f8-bc6e-4ae5-b5e2-533007b38898 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfb3afb3441f4676b201ab93c9dfdefe', 'request_id': 'req-e4a314f8-bc6e-4ae5-b5e2-533007b38898'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:11.422777 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-e4a314f8-bc6e-4ae5-b5e2-533007b38898 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfb3afb3441f4676b201ab93c9dfdefe', 'request_id': 'req-e4a314f8-bc6e-4ae5-b5e2-533007b38898'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:11.450340 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-e4a314f8-bc6e-4ae5-b5e2-533007b38898 admin admin] The token cache is being invalidate because project dfb3afb3441f4676b201ab93c9dfdefe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81618) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:15:11.451442 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-e4a314f8-bc6e-4ae5-b5e2-533007b38898 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:11.452714 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-e4a314f8-bc6e-4ae5-b5e2-533007b38898 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:11.456145 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 998/1994] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:15:11 2022] DELETE /identity/v3/projects/dfb3afb3441f4676b201ab93c9dfdefe => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:15:11.467669 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fd36f7-2509-49ce-a92a-6e221ce5272e None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:11.468657 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fd36f7-2509-49ce-a92a-6e221ce5272e None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:11.469192 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fd36f7-2509-49ce-a92a-6e221ce5272e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:11.566991 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-f8fd36f7-2509-49ce-a92a-6e221ce5272e None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:15:11.571324 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-f8fd36f7-2509-49ce-a92a-6e221ce5272e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:15:11.618749 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f462e2c0-6eb6-4eb2-9e1a-bf530737f5a5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5632ade0cc6f4ae28f72160de68486cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:11.619819 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f462e2c0-6eb6-4eb2-9e1a-bf530737f5a5 service glance] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:11.620294 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f462e2c0-6eb6-4eb2-9e1a-bf530737f5a5 service glance] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:11.620767 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f462e2c0-6eb6-4eb2-9e1a-bf530737f5a5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:11.624615 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-f462e2c0-6eb6-4eb2-9e1a-bf530737f5a5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:15:11.824505 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 999/1995] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:15:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:15:11.839503 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e6b39c-f382-4628-b3fa-e573b6855cf3 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:11.839802 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e6b39c-f382-4628-b3fa-e573b6855cf3 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:11.840121 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e6b39c-f382-4628-b3fa-e573b6855cf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:11.853504 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f462e2c0-6eb6-4eb2-9e1a-bf530737f5a5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:11.857050 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f462e2c0-6eb6-4eb2-9e1a-bf530737f5a5 service glance] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:11.868360 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:15:11.873494 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 997/1996] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:15:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 643 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:15:11.877729 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-74e6b39c-f382-4628-b3fa-e573b6855cf3 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:15:11.884310 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-74e6b39c-f382-4628-b3fa-e573b6855cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:15:11.905420 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1000/1997] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:15:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:15:11.915149 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f462e2c0-6eb6-4eb2-9e1a-bf530737f5a5 service glance] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:11.930107 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f462e2c0-6eb6-4eb2-9e1a-bf530737f5a5 service glance] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:11.936825 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f558e4b-d9e2-4371-a3b7-2150a1cdc8ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:11.938004 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f558e4b-d9e2-4371-a3b7-2150a1cdc8ee admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:11.938595 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f558e4b-d9e2-4371-a3b7-2150a1cdc8ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:11.939083 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f558e4b-d9e2-4371-a3b7-2150a1cdc8ee admin admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:11.940251 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f558e4b-d9e2-4371-a3b7-2150a1cdc8ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:15:11.941075 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f558e4b-d9e2-4371-a3b7-2150a1cdc8ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:11.944580 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f558e4b-d9e2-4371-a3b7-2150a1cdc8ee admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:11.966427 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 998/1998] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:15:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:15:11.971321 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e6b39c-f382-4628-b3fa-e573b6855cf3 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:11.986429 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e6b39c-f382-4628-b3fa-e573b6855cf3 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:11.993242 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2ea99c-07be-4f6d-86ed-c155fa759f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:11.993862 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ea99c-07be-4f6d-86ed-c155fa759f09 admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:11.994073 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ea99c-07be-4f6d-86ed-c155fa759f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:11.994413 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ea99c-07be-4f6d-86ed-c155fa759f09 admin admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:11.995053 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2ea99c-07be-4f6d-86ed-c155fa759f09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:11.997606 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2ea99c-07be-4f6d-86ed-c155fa759f09 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:12.038138 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1001/1999] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:15:11 2022] POST /identity/v3/projects => generated 377 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:15:12.044750 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f558e4b-d9e2-4371-a3b7-2150a1cdc8ee admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:12.059277 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f558e4b-d9e2-4371-a3b7-2150a1cdc8ee admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:12.065352 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:15:12.152556 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c805b3b2-0881-4de3-a5c2-6b5315e7799d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:12.153542 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c805b3b2-0881-4de3-a5c2-6b5315e7799d admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:12.156482 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c805b3b2-0881-4de3-a5c2-6b5315e7799d admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:12.156482 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c805b3b2-0881-4de3-a5c2-6b5315e7799d admin admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:12.157399 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c805b3b2-0881-4de3-a5c2-6b5315e7799d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:12.160583 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c805b3b2-0881-4de3-a5c2-6b5315e7799d admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:12.194797 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 999/2000] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:15:12 2022] POST /identity/v3/users => generated 289 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:15:12.199414 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2ea99c-07be-4f6d-86ed-c155fa759f09 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:12.213906 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2ea99c-07be-4f6d-86ed-c155fa759f09 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:12.223123 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec884b0d-31f4-471c-8569-d3f63e347e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:12.224896 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec884b0d-31f4-471c-8569-d3f63e347e7c admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:12.225510 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec884b0d-31f4-471c-8569-d3f63e347e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:12.226128 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec884b0d-31f4-471c-8569-d3f63e347e7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:12.227461 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec884b0d-31f4-471c-8569-d3f63e347e7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:12.231083 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec884b0d-31f4-471c-8569-d3f63e347e7c admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:12.253070 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1002/2001] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:15:12 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:15:12.255183 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c805b3b2-0881-4de3-a5c2-6b5315e7799d admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:12.289312 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c805b3b2-0881-4de3-a5c2-6b5315e7799d admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:12.298095 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6a72a2-6ee0-472e-9801-a546864b7193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:12.299122 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6a72a2-6ee0-472e-9801-a546864b7193 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:12.299772 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6a72a2-6ee0-472e-9801-a546864b7193 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:12.300584 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6a72a2-6ee0-472e-9801-a546864b7193 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:12.302836 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6a72a2-6ee0-472e-9801-a546864b7193 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:12.307143 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6a72a2-6ee0-472e-9801-a546864b7193 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:12.327276 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1000/2002] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:15:12 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 16:15:12.329489 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec884b0d-31f4-471c-8569-d3f63e347e7c admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:12.343366 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec884b0d-31f4-471c-8569-d3f63e347e7c admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:12.349664 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8a27fc-ecee-4b2d-b00f-42ac0ef57c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:12.350494 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a27fc-ecee-4b2d-b00f-42ac0ef57c79 admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:12.350832 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a27fc-ecee-4b2d-b00f-42ac0ef57c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:12.351186 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a27fc-ecee-4b2d-b00f-42ac0ef57c79 admin admin] PATH_INFO: `/v3/projects/2a726afa4ec84ec99a5b1bd08c95a0cd/users/9fd6c22a61be43658ae87052b867e384/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:12.369349 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8a27fc-ecee-4b2d-b00f-42ac0ef57c79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a726afa4ec84ec99a5b1bd08c95a0cd, user_id=9fd6c22a61be43658ae87052b867e384, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:12.372796 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8a27fc-ecee-4b2d-b00f-42ac0ef57c79 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:12.390194 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1003/2003] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:15:12 2022] PUT /identity/v3/projects/2a726afa4ec84ec99a5b1bd08c95a0cd/users/9fd6c22a61be43658ae87052b867e384/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 16:15:12.396159 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f767ce0a-d89c-4da9-96cf-c64eb9f7d23b None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:12.396557 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f767ce0a-d89c-4da9-96cf-c64eb9f7d23b None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:12.396929 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f767ce0a-d89c-4da9-96cf-c64eb9f7d23b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:12.426729 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-f767ce0a-d89c-4da9-96cf-c64eb9f7d23b None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:15:12.428628 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-f767ce0a-d89c-4da9-96cf-c64eb9f7d23b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:15:12.561350 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1001/2004] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:15:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:15:12.572902 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8a27fc-ecee-4b2d-b00f-42ac0ef57c79 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:12.575094 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-3a8a27fc-ecee-4b2d-b00f-42ac0ef57c79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:15:12.670839 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8a27fc-ecee-4b2d-b00f-42ac0ef57c79 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:12.675619 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c34b0c4-8c8a-46ff-b479-a5b345953419 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:12.676387 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c34b0c4-8c8a-46ff-b479-a5b345953419 service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:12.677249 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c34b0c4-8c8a-46ff-b479-a5b345953419 service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:12.677619 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c34b0c4-8c8a-46ff-b479-a5b345953419 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:12.702988 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c34b0c4-8c8a-46ff-b479-a5b345953419 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:12.704444 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c34b0c4-8c8a-46ff-b479-a5b345953419 service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:12.715322 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1004/2005] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:15:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 16:15:17.857093 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4828a9c-961a-49ae-80c2-21b1d203552a None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:17.857982 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4828a9c-961a-49ae-80c2-21b1d203552a None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:17.857982 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4828a9c-961a-49ae-80c2-21b1d203552a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:17.934933 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-c4828a9c-961a-49ae-80c2-21b1d203552a None None] MFA Rules not processed for user `fe870f5b19334b47b972d7655876472e`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:15:17.939978 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-c4828a9c-961a-49ae-80c2-21b1d203552a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:15:18.134197 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1002/2006] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:15:17 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:15:21.599199 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c9db93-c4e5-4d24-891f-a076e6329c82 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:21.600114 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c9db93-c4e5-4d24-891f-a076e6329c82 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:21.600114 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c9db93-c4e5-4d24-891f-a076e6329c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:21.665382 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-23c9db93-c4e5-4d24-891f-a076e6329c82 None None] MFA Rules not processed for user `9fd6c22a61be43658ae87052b867e384`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:15:21.667177 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-23c9db93-c4e5-4d24-891f-a076e6329c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:15:21.913496 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1005/2007] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:15:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:15:21.920675 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4828a9c-961a-49ae-80c2-21b1d203552a None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:21.940744 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4828a9c-961a-49ae-80c2-21b1d203552a None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:21.952017 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81446f84-f1ab-418a-956c-41c3232cfb19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:21.953307 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81446f84-f1ab-418a-956c-41c3232cfb19 admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:21.953810 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81446f84-f1ab-418a-956c-41c3232cfb19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:21.954309 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81446f84-f1ab-418a-956c-41c3232cfb19 admin admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:21.955983 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81446f84-f1ab-418a-956c-41c3232cfb19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:21.959919 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81446f84-f1ab-418a-956c-41c3232cfb19 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:22.049053 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1003/2008] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:15:21 2022] POST /identity/v3/projects => generated 377 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 16:15:22.055742 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c9db93-c4e5-4d24-891f-a076e6329c82 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:22.080451 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c9db93-c4e5-4d24-891f-a076e6329c82 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:22.132558 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81446f84-f1ab-418a-956c-41c3232cfb19 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:22.135310 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-81446f84-f1ab-418a-956c-41c3232cfb19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:15:22.178350 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:15:22.178568 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:15:22.317582 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39f1426-bfc9-462a-91b1-7d8a296bbfdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:22.318474 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39f1426-bfc9-462a-91b1-7d8a296bbfdb admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:22.318887 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39f1426-bfc9-462a-91b1-7d8a296bbfdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:22.319261 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39f1426-bfc9-462a-91b1-7d8a296bbfdb admin admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:22.320042 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39f1426-bfc9-462a-91b1-7d8a296bbfdb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:22.323678 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39f1426-bfc9-462a-91b1-7d8a296bbfdb admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:22.343185 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81446f84-f1ab-418a-956c-41c3232cfb19 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:22.350712 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9bfe0b-a5df-4ac5-b63b-8de7c95cc97a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:22.351674 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9bfe0b-a5df-4ac5-b63b-8de7c95cc97a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:22.351965 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9bfe0b-a5df-4ac5-b63b-8de7c95cc97a admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:22.352209 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9bfe0b-a5df-4ac5-b63b-8de7c95cc97a admin admin] PATH_INFO: `/v3/users/428a95af27d2468bb39391ca3eefea40` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:22.353471 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9bfe0b-a5df-4ac5-b63b-8de7c95cc97a admin admin] RBAC: Authorizing `identity:delete_user(user_id=428a95af27d2468bb39391ca3eefea40)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:22.355531 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9bfe0b-a5df-4ac5-b63b-8de7c95cc97a admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:22.373400 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1006/2009] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:15:22 2022] POST /identity/v3/users => generated 295 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:15:22.375021 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39f1426-bfc9-462a-91b1-7d8a296bbfdb admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:22.425773 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39f1426-bfc9-462a-91b1-7d8a296bbfdb admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:22.447016 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fea8856-e3b0-4b81-89da-b752ce78d27b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:22.452817 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fea8856-e3b0-4b81-89da-b752ce78d27b admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:22.453597 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fea8856-e3b0-4b81-89da-b752ce78d27b admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:22.455714 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fea8856-e3b0-4b81-89da-b752ce78d27b admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:22.459012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fea8856-e3b0-4b81-89da-b752ce78d27b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:22.468752 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fea8856-e3b0-4b81-89da-b752ce78d27b admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:22.479229 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-af9bfe0b-a5df-4ac5-b63b-8de7c95cc97a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '428a95af27d2468bb39391ca3eefea40'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:22.482016 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-af9bfe0b-a5df-4ac5-b63b-8de7c95cc97a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '428a95af27d2468bb39391ca3eefea40'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:22.500396 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1007/2010] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:15:22 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:15:22.500684 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-af9bfe0b-a5df-4ac5-b63b-8de7c95cc97a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '428a95af27d2468bb39391ca3eefea40'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:22.503112 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fea8856-e3b0-4b81-89da-b752ce78d27b admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:22.507191 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-0fea8856-e3b0-4b81-89da-b752ce78d27b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:15:22.563634 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fea8856-e3b0-4b81-89da-b752ce78d27b admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:22.577194 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-af9bfe0b-a5df-4ac5-b63b-8de7c95cc97a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '428a95af27d2468bb39391ca3eefea40'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:22.580421 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-af9bfe0b-a5df-4ac5-b63b-8de7c95cc97a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '428a95af27d2468bb39391ca3eefea40'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:22.584508 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a484050d-84a7-4916-936a-461dc1e036e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:22.589672 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a484050d-84a7-4916-936a-461dc1e036e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:22.590557 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a484050d-84a7-4916-936a-461dc1e036e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:22.593638 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a484050d-84a7-4916-936a-461dc1e036e0 admin admin] PATH_INFO: `/v3/projects/7b1b5a5be7314be08493869c523266bf/users/a8d61d084bf948679e9f849bbd743bc6/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:22.638494 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1004/2011] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:15:22 2022] DELETE /identity/v3/users/428a95af27d2468bb39391ca3eefea40 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:15:22.661696 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a484050d-84a7-4916-936a-461dc1e036e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b1b5a5be7314be08493869c523266bf, user_id=a8d61d084bf948679e9f849bbd743bc6, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:22.666569 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a484050d-84a7-4916-936a-461dc1e036e0 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:22.692621 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1008/2012] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:15:22 2022] PUT /identity/v3/projects/7b1b5a5be7314be08493869c523266bf/users/a8d61d084bf948679e9f849bbd743bc6/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:15:22.698797 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9bfe0b-a5df-4ac5-b63b-8de7c95cc97a admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:22.804489 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9bfe0b-a5df-4ac5-b63b-8de7c95cc97a admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:22.879443 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:15:22.936901 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3fd308-69d5-4e8d-af6f-79275e1be7c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:22.937690 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3fd308-69d5-4e8d-af6f-79275e1be7c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:22.938144 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3fd308-69d5-4e8d-af6f-79275e1be7c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:22.938516 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3fd308-69d5-4e8d-af6f-79275e1be7c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:22.939537 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3fd308-69d5-4e8d-af6f-79275e1be7c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:22.943427 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3fd308-69d5-4e8d-af6f-79275e1be7c3 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:22.961144 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1005/2013] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:15:22 2022] GET /identity/v3/roles => generated 1420 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:15:22.962345 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a484050d-84a7-4916-936a-461dc1e036e0 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:22.975199 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a484050d-84a7-4916-936a-461dc1e036e0 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:22.982088 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dedf57b-50c8-41ec-9777-6d5423e79cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:22.983023 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dedf57b-50c8-41ec-9777-6d5423e79cbe admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:22.983403 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dedf57b-50c8-41ec-9777-6d5423e79cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:22.983778 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dedf57b-50c8-41ec-9777-6d5423e79cbe admin admin] PATH_INFO: `/v3/projects/7b1b5a5be7314be08493869c523266bf/users/a8d61d084bf948679e9f849bbd743bc6/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:22.986509 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dedf57b-50c8-41ec-9777-6d5423e79cbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b1b5a5be7314be08493869c523266bf, user_id=a8d61d084bf948679e9f849bbd743bc6, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:22.996234 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dedf57b-50c8-41ec-9777-6d5423e79cbe admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:23.034365 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1009/2014] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:15:22 2022] PUT /identity/v3/projects/7b1b5a5be7314be08493869c523266bf/users/a8d61d084bf948679e9f849bbd743bc6/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:15:23.235260 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3fd308-69d5-4e8d-af6f-79275e1be7c3 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:23.237359 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-7e3fd308-69d5-4e8d-af6f-79275e1be7c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:15:23.253029 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3fd308-69d5-4e8d-af6f-79275e1be7c3 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:23.425899 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff46561-333e-495f-9853-7bb8f2479c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:23.426828 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff46561-333e-495f-9853-7bb8f2479c63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:23.427255 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff46561-333e-495f-9853-7bb8f2479c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:23.427618 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff46561-333e-495f-9853-7bb8f2479c63 admin admin] PATH_INFO: `/v3/projects/34baf55057ee42ffbd099de2d65544a1` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:23.429538 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff46561-333e-495f-9853-7bb8f2479c63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34baf55057ee42ffbd099de2d65544a1)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:23.432951 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff46561-333e-495f-9853-7bb8f2479c63 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:23.566433 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-7ff46561-333e-495f-9853-7bb8f2479c63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34baf55057ee42ffbd099de2d65544a1', 'request_id': 'req-7ff46561-333e-495f-9853-7bb8f2479c63'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:23.593065 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-7ff46561-333e-495f-9853-7bb8f2479c63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34baf55057ee42ffbd099de2d65544a1', 'request_id': 'req-7ff46561-333e-495f-9853-7bb8f2479c63'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:23.611556 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-7ff46561-333e-495f-9853-7bb8f2479c63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34baf55057ee42ffbd099de2d65544a1', 'request_id': 'req-7ff46561-333e-495f-9853-7bb8f2479c63'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:23.643841 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-7ff46561-333e-495f-9853-7bb8f2479c63 admin admin] The token cache is being invalidate because project 34baf55057ee42ffbd099de2d65544a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81617) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:15:23.644346 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-7ff46561-333e-495f-9853-7bb8f2479c63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:23.645564 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-7ff46561-333e-495f-9853-7bb8f2479c63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:23.652516 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1006/2015] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:15:23 2022] DELETE /identity/v3/projects/34baf55057ee42ffbd099de2d65544a1 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:15:23.655664 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b173ae63-ad55-491a-b742-8a05c6634c47 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:23.656162 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b173ae63-ad55-491a-b742-8a05c6634c47 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:23.656586 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b173ae63-ad55-491a-b742-8a05c6634c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:23.686657 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-b173ae63-ad55-491a-b742-8a05c6634c47 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:15:23.689204 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-b173ae63-ad55-491a-b742-8a05c6634c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:15:23.913831 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1010/2016] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:15:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:15:23.923197 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12910301-a018-4fdf-a639-84cb6192ce99 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:23.923624 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12910301-a018-4fdf-a639-84cb6192ce99 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:23.924051 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12910301-a018-4fdf-a639-84cb6192ce99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:23.979624 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-12910301-a018-4fdf-a639-84cb6192ce99 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:15:23.983109 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-12910301-a018-4fdf-a639-84cb6192ce99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:15:24.009952 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1007/2017] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:15:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:15:24.011759 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b173ae63-ad55-491a-b742-8a05c6634c47 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:24.025947 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b173ae63-ad55-491a-b742-8a05c6634c47 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:24.031846 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54913257-5a49-438a-8f64-9e5b1608c6f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:24.032621 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54913257-5a49-438a-8f64-9e5b1608c6f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:24.032980 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54913257-5a49-438a-8f64-9e5b1608c6f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:24.033330 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54913257-5a49-438a-8f64-9e5b1608c6f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:24.034093 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54913257-5a49-438a-8f64-9e5b1608c6f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:15:24.034542 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54913257-5a49-438a-8f64-9e5b1608c6f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:24.036531 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54913257-5a49-438a-8f64-9e5b1608c6f9 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:24.053356 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1011/2018] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:15:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:15:24.058692 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12910301-a018-4fdf-a639-84cb6192ce99 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:24.072001 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12910301-a018-4fdf-a639-84cb6192ce99 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:24.094134 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976f6f22-91ae-41ae-9f7f-9c5bf75eaaf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:24.094808 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976f6f22-91ae-41ae-9f7f-9c5bf75eaaf3 admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:24.095093 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976f6f22-91ae-41ae-9f7f-9c5bf75eaaf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:24.095373 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976f6f22-91ae-41ae-9f7f-9c5bf75eaaf3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:24.096051 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976f6f22-91ae-41ae-9f7f-9c5bf75eaaf3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:24.099094 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976f6f22-91ae-41ae-9f7f-9c5bf75eaaf3 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:24.171868 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1008/2019] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:15:24 2022] POST /identity/v3/projects => generated 373 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:15:24.176135 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54913257-5a49-438a-8f64-9e5b1608c6f9 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:24.192889 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54913257-5a49-438a-8f64-9e5b1608c6f9 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:24.309380 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e80964b-628e-4538-a222-ea8113c323c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:24.310196 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e80964b-628e-4538-a222-ea8113c323c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:24.310651 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e80964b-628e-4538-a222-ea8113c323c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:24.311120 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e80964b-628e-4538-a222-ea8113c323c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:24.312269 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e80964b-628e-4538-a222-ea8113c323c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:24.315963 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e80964b-628e-4538-a222-ea8113c323c9 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:24.330104 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:15:24.366460 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1012/2020] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:15:24 2022] POST /identity/v3/users => generated 287 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:15:24.368843 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976f6f22-91ae-41ae-9f7f-9c5bf75eaaf3 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:24.398791 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976f6f22-91ae-41ae-9f7f-9c5bf75eaaf3 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:24.413333 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1064e33e-6ec8-4e50-86ee-057159e196a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:24.413987 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1064e33e-6ec8-4e50-86ee-057159e196a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:24.414266 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1064e33e-6ec8-4e50-86ee-057159e196a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:24.414572 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1064e33e-6ec8-4e50-86ee-057159e196a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:24.415250 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1064e33e-6ec8-4e50-86ee-057159e196a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:24.416917 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1064e33e-6ec8-4e50-86ee-057159e196a1 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:24.457371 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1009/2021] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:15:24 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 16:15:24.463101 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e80964b-628e-4538-a222-ea8113c323c9 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:24.480388 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e80964b-628e-4538-a222-ea8113c323c9 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:24.496908 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a754e58c-de77-4471-89a8-47355eff956f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:24.497704 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a754e58c-de77-4471-89a8-47355eff956f admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:24.498189 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a754e58c-de77-4471-89a8-47355eff956f admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:24.498725 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a754e58c-de77-4471-89a8-47355eff956f admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:24.500658 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a754e58c-de77-4471-89a8-47355eff956f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:24.506036 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a754e58c-de77-4471-89a8-47355eff956f admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:24.556220 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1013/2022] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:15:24 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:15:24.557654 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1064e33e-6ec8-4e50-86ee-057159e196a1 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:24.587791 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1064e33e-6ec8-4e50-86ee-057159e196a1 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:24.595041 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7f4cfb-ec04-47c6-a34a-b242a618cf27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:24.596371 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f4cfb-ec04-47c6-a34a-b242a618cf27 admin admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:24.596918 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f4cfb-ec04-47c6-a34a-b242a618cf27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:24.597348 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f4cfb-ec04-47c6-a34a-b242a618cf27 admin admin] PATH_INFO: `/v3/projects/48dbc60f791341f9a8d3e68ea865ce27/users/d3fc9058ff4f484fa592228411954b71/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:24.628947 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7f4cfb-ec04-47c6-a34a-b242a618cf27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48dbc60f791341f9a8d3e68ea865ce27, user_id=d3fc9058ff4f484fa592228411954b71, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:24.635865 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7f4cfb-ec04-47c6-a34a-b242a618cf27 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:24.654393 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:15:24.675533 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5159c14-6ff4-4ea6-a092-239fd241a627 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:24.676004 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1010/2023] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:15:24 2022] PUT /identity/v3/projects/48dbc60f791341f9a8d3e68ea865ce27/users/d3fc9058ff4f484fa592228411954b71/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:15:24.677447 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5159c14-6ff4-4ea6-a092-239fd241a627 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:24.678768 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5159c14-6ff4-4ea6-a092-239fd241a627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:24.747664 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-e5159c14-6ff4-4ea6-a092-239fd241a627 None None] MFA Rules not processed for user `d3fc9058ff4f484fa592228411954b71`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:15:24.750084 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-e5159c14-6ff4-4ea6-a092-239fd241a627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:15:24.984395 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1014/2024] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:15:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:15:24.993611 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7f4cfb-ec04-47c6-a34a-b242a618cf27 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:25.181001 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7f4cfb-ec04-47c6-a34a-b242a618cf27 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:25.190697 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79b0ee8-2afe-43f1-821a-3222f75193a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:25.191768 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79b0ee8-2afe-43f1-821a-3222f75193a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:25.193516 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79b0ee8-2afe-43f1-821a-3222f75193a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:25.194816 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79b0ee8-2afe-43f1-821a-3222f75193a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:25.196584 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79b0ee8-2afe-43f1-821a-3222f75193a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:25.200432 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79b0ee8-2afe-43f1-821a-3222f75193a0 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:25.266894 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1011/2025] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:15:24 2022] POST /identity/v3/projects => generated 371 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:15:25.267257 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5159c14-6ff4-4ea6-a092-239fd241a627 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:25.283301 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5159c14-6ff4-4ea6-a092-239fd241a627 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:25.429084 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a2f063-5624-4cd1-903d-53300db94aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:25.429983 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a2f063-5624-4cd1-903d-53300db94aac admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:25.430543 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a2f063-5624-4cd1-903d-53300db94aac admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:25.432377 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a2f063-5624-4cd1-903d-53300db94aac admin admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:25.435559 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a2f063-5624-4cd1-903d-53300db94aac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:25.440730 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a2f063-5624-4cd1-903d-53300db94aac admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:25.489332 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1015/2026] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:15:25 2022] POST /identity/v3/users => generated 292 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 16:15:25.493588 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79b0ee8-2afe-43f1-821a-3222f75193a0 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:25.508679 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79b0ee8-2afe-43f1-821a-3222f75193a0 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:25.521660 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8df1d5-4ea8-4629-aeab-ac8296e63a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:25.522830 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8df1d5-4ea8-4629-aeab-ac8296e63a03 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:25.523412 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8df1d5-4ea8-4629-aeab-ac8296e63a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:25.524711 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8df1d5-4ea8-4629-aeab-ac8296e63a03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:25.527395 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8df1d5-4ea8-4629-aeab-ac8296e63a03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:25.531727 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8df1d5-4ea8-4629-aeab-ac8296e63a03 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:25.561232 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1012/2027] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:15:25.565959 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a2f063-5624-4cd1-903d-53300db94aac admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:25.579825 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a2f063-5624-4cd1-903d-53300db94aac admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:25.590444 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e9a2af-4f0f-41fd-b85f-90880deb9a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:25.591172 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e9a2af-4f0f-41fd-b85f-90880deb9a2a admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:25.591457 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e9a2af-4f0f-41fd-b85f-90880deb9a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:25.591777 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e9a2af-4f0f-41fd-b85f-90880deb9a2a admin admin] PATH_INFO: `/v3/projects/90a66807e477439a9b1c3df49ee7d5ff/users/e1ffcd8687cc451fbb84b849e592693e/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:25.618800 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e9a2af-4f0f-41fd-b85f-90880deb9a2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=90a66807e477439a9b1c3df49ee7d5ff, user_id=e1ffcd8687cc451fbb84b849e592693e, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:25.623011 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e9a2af-4f0f-41fd-b85f-90880deb9a2a admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:25.647137 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1016/2028] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:15:25 2022] PUT /identity/v3/projects/90a66807e477439a9b1c3df49ee7d5ff/users/e1ffcd8687cc451fbb84b849e592693e/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:15:25.651631 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8df1d5-4ea8-4629-aeab-ac8296e63a03 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:25.679957 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8df1d5-4ea8-4629-aeab-ac8296e63a03 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:25.907046 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8804e1d-0708-4826-a8f7-1e43adebee1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:25.908306 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8804e1d-0708-4826-a8f7-1e43adebee1f admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:25.908814 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8804e1d-0708-4826-a8f7-1e43adebee1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:25.912197 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8804e1d-0708-4826-a8f7-1e43adebee1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:25.913644 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8804e1d-0708-4826-a8f7-1e43adebee1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:25.921836 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8804e1d-0708-4826-a8f7-1e43adebee1f admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:25.973976 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1013/2029] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:15:25.978611 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e9a2af-4f0f-41fd-b85f-90880deb9a2a admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:26.001395 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e9a2af-4f0f-41fd-b85f-90880deb9a2a admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:26.018106 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceb32b4-00a7-445b-9968-534a5d1be0d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:26.019424 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceb32b4-00a7-445b-9968-534a5d1be0d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:26.019918 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceb32b4-00a7-445b-9968-534a5d1be0d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:26.020466 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceb32b4-00a7-445b-9968-534a5d1be0d0 admin admin] PATH_INFO: `/v3/projects/90a66807e477439a9b1c3df49ee7d5ff/users/e1ffcd8687cc451fbb84b849e592693e/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:26.026527 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceb32b4-00a7-445b-9968-534a5d1be0d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90a66807e477439a9b1c3df49ee7d5ff, user_id=e1ffcd8687cc451fbb84b849e592693e, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:26.037858 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceb32b4-00a7-445b-9968-534a5d1be0d0 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:26.089376 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1017/2030] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:15:25 2022] PUT /identity/v3/projects/90a66807e477439a9b1c3df49ee7d5ff/users/e1ffcd8687cc451fbb84b849e592693e/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:15:26.111619 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797e506-5f06-41c6-b459-eeedd335f037 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:26.118468 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797e506-5f06-41c6-b459-eeedd335f037 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:26.118468 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797e506-5f06-41c6-b459-eeedd335f037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:26.255714 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-9797e506-5f06-41c6-b459-eeedd335f037 None None] MFA Rules not processed for user `e1ffcd8687cc451fbb84b849e592693e`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:15:26.265815 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-9797e506-5f06-41c6-b459-eeedd335f037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:15:26.546458 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:15:26.587504 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1014/2031] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:15:26 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 478 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:15:26.614419 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceb32b4-00a7-445b-9968-534a5d1be0d0 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:26.618236 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-fceb32b4-00a7-445b-9968-534a5d1be0d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:15:26.794459 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:15:26.897199 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceb32b4-00a7-445b-9968-534a5d1be0d0 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:26.912747 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed3f549-0d96-4023-b5cf-2ea2521bc3d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:26.914554 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed3f549-0d96-4023-b5cf-2ea2521bc3d5 service cinder] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:26.914554 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed3f549-0d96-4023-b5cf-2ea2521bc3d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:26.915957 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed3f549-0d96-4023-b5cf-2ea2521bc3d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:26.955594 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed3f549-0d96-4023-b5cf-2ea2521bc3d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:26.958992 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed3f549-0d96-4023-b5cf-2ea2521bc3d5 service cinder] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:26.997970 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1018/2032] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:15:26 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:15:27.267361 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9797e506-5f06-41c6-b459-eeedd335f037 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:27.290999 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9797e506-5f06-41c6-b459-eeedd335f037 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:27.299208 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7727a8-7e19-47eb-855c-778fa25d1b81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:27.302471 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7727a8-7e19-47eb-855c-778fa25d1b81 service cinder] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:27.302825 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7727a8-7e19-47eb-855c-778fa25d1b81 service cinder] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:27.303103 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7727a8-7e19-47eb-855c-778fa25d1b81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:27.493936 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7727a8-7e19-47eb-855c-778fa25d1b81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:27.497559 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7727a8-7e19-47eb-855c-778fa25d1b81 service cinder] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:27.519623 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1015/2033] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:15:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 16:15:32.522865 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546aad1f-4070-465c-bc7a-2298074330de None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:32.523552 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546aad1f-4070-465c-bc7a-2298074330de None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:32.523552 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546aad1f-4070-465c-bc7a-2298074330de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:32.673456 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-546aad1f-4070-465c-bc7a-2298074330de None None] MFA Rules not processed for user `a8d61d084bf948679e9f849bbd743bc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:15:32.674766 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-546aad1f-4070-465c-bc7a-2298074330de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:15:32.883478 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1019/2034] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:15:32 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:15:32.891042 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7727a8-7e19-47eb-855c-778fa25d1b81 service cinder] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:32.893228 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-4c7727a8-7e19-47eb-855c-778fa25d1b81 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:15:33.052556 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7727a8-7e19-47eb-855c-778fa25d1b81 service cinder] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:33.062873 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bcec2d-a304-4db0-b7d6-6d9e836d6821 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:33.063779 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bcec2d-a304-4db0-b7d6-6d9e836d6821 service nova] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:33.067703 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bcec2d-a304-4db0-b7d6-6d9e836d6821 service nova] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:33.067703 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bcec2d-a304-4db0-b7d6-6d9e836d6821 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:33.074411 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-23bcec2d-a304-4db0-b7d6-6d9e836d6821 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:15:33.189416 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bcec2d-a304-4db0-b7d6-6d9e836d6821 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:33.192973 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bcec2d-a304-4db0-b7d6-6d9e836d6821 service nova] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:33.210875 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1016/2035] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:15:32 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 16:15:35.828343 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546aad1f-4070-465c-bc7a-2298074330de None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:35.832468 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-546aad1f-4070-465c-bc7a-2298074330de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:15:36.172426 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546aad1f-4070-465c-bc7a-2298074330de None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:36.214247 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73db1b0c-f225-4244-8567-70cbc0575388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:36.215409 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db1b0c-f225-4244-8567-70cbc0575388 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:36.215760 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db1b0c-f225-4244-8567-70cbc0575388 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:36.219429 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db1b0c-f225-4244-8567-70cbc0575388 admin admin] PATH_INFO: `/v3/users/5c397ef8d5b34358a6afeb85ea551006` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:36.224763 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73db1b0c-f225-4244-8567-70cbc0575388 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c397ef8d5b34358a6afeb85ea551006)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:36.230043 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73db1b0c-f225-4244-8567-70cbc0575388 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:36.394914 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-73db1b0c-f225-4244-8567-70cbc0575388 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c397ef8d5b34358a6afeb85ea551006'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:36.446263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:15:36.463258 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-73db1b0c-f225-4244-8567-70cbc0575388 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c397ef8d5b34358a6afeb85ea551006'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:36.503320 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-73db1b0c-f225-4244-8567-70cbc0575388 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c397ef8d5b34358a6afeb85ea551006'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:36.522209 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-73db1b0c-f225-4244-8567-70cbc0575388 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c397ef8d5b34358a6afeb85ea551006'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:36.524402 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-73db1b0c-f225-4244-8567-70cbc0575388 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c397ef8d5b34358a6afeb85ea551006'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:36.540252 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1020/2036] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:15:35 2022] DELETE /identity/v3/users/5c397ef8d5b34358a6afeb85ea551006 => generated 0 bytes in 713 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:15:36.545659 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa3686-89b0-48a5-9d86-a817a6529bc7 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:36.546554 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa3686-89b0-48a5-9d86-a817a6529bc7 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:36.547006 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa3686-89b0-48a5-9d86-a817a6529bc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:36.595556 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-fafa3686-89b0-48a5-9d86-a817a6529bc7 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:15:36.596995 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-fafa3686-89b0-48a5-9d86-a817a6529bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:15:36.712261 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:15:36.826334 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1017/2037] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:15:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:15:36.840194 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73db1b0c-f225-4244-8567-70cbc0575388 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:36.842241 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-73db1b0c-f225-4244-8567-70cbc0575388 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:15:36.969328 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73db1b0c-f225-4244-8567-70cbc0575388 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:36.983655 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffadb13-05d6-4227-b480-6661c9155864 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:36.985086 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffadb13-05d6-4227-b480-6661c9155864 service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:36.985542 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffadb13-05d6-4227-b480-6661c9155864 service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:36.985996 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffadb13-05d6-4227-b480-6661c9155864 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:37.020656 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffadb13-05d6-4227-b480-6661c9155864 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:37.023550 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffadb13-05d6-4227-b480-6661c9155864 service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:37.051021 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1021/2038] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:15:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:15:37.823747 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafa3686-89b0-48a5-9d86-a817a6529bc7 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:37.826312 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-fafa3686-89b0-48a5-9d86-a817a6529bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:15:37.886920 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafa3686-89b0-48a5-9d86-a817a6529bc7 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:37.897614 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aa0a4e-3e0b-4ca3-9ab5-59f8c910c0a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:37.899110 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa0a4e-3e0b-4ca3-9ab5-59f8c910c0a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:37.899647 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa0a4e-3e0b-4ca3-9ab5-59f8c910c0a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:37.900099 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa0a4e-3e0b-4ca3-9ab5-59f8c910c0a6 admin admin] PATH_INFO: `/v3/projects/35121c2a50974cbfab5bd219a1b164ec` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:37.901861 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aa0a4e-3e0b-4ca3-9ab5-59f8c910c0a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35121c2a50974cbfab5bd219a1b164ec)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:37.904871 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aa0a4e-3e0b-4ca3-9ab5-59f8c910c0a6 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:37.975286 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-f0aa0a4e-3e0b-4ca3-9ab5-59f8c910c0a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35121c2a50974cbfab5bd219a1b164ec', 'request_id': 'req-f0aa0a4e-3e0b-4ca3-9ab5-59f8c910c0a6'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:37.987068 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-f0aa0a4e-3e0b-4ca3-9ab5-59f8c910c0a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35121c2a50974cbfab5bd219a1b164ec', 'request_id': 'req-f0aa0a4e-3e0b-4ca3-9ab5-59f8c910c0a6'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:37.998878 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-f0aa0a4e-3e0b-4ca3-9ab5-59f8c910c0a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35121c2a50974cbfab5bd219a1b164ec', 'request_id': 'req-f0aa0a4e-3e0b-4ca3-9ab5-59f8c910c0a6'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:38.019477 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-f0aa0a4e-3e0b-4ca3-9ab5-59f8c910c0a6 admin admin] The token cache is being invalidate because project 35121c2a50974cbfab5bd219a1b164ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81617) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:15:38.019829 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-f0aa0a4e-3e0b-4ca3-9ab5-59f8c910c0a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:38.020557 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-f0aa0a4e-3e0b-4ca3-9ab5-59f8c910c0a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:38.026164 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1018/2039] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:15:37 2022] DELETE /identity/v3/projects/35121c2a50974cbfab5bd219a1b164ec => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:15:38.030828 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aee234-b5e1-415b-98f6-fe06f9777bcd None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:38.031966 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aee234-b5e1-415b-98f6-fe06f9777bcd None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:38.032349 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aee234-b5e1-415b-98f6-fe06f9777bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:38.070399 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-51aee234-b5e1-415b-98f6-fe06f9777bcd None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:15:38.073073 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-51aee234-b5e1-415b-98f6-fe06f9777bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:15:38.202638 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1022/2040] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:15:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:15:38.208679 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea83be3-38d3-40fd-b7bc-3d343cb7c4bf None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:38.212406 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea83be3-38d3-40fd-b7bc-3d343cb7c4bf None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:38.212907 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea83be3-38d3-40fd-b7bc-3d343cb7c4bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:38.296061 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-4ea83be3-38d3-40fd-b7bc-3d343cb7c4bf None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:15:38.298031 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-4ea83be3-38d3-40fd-b7bc-3d343cb7c4bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:15:38.375404 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1019/2041] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:15:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:15:38.382164 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aee234-b5e1-415b-98f6-fe06f9777bcd None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:38.398666 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aee234-b5e1-415b-98f6-fe06f9777bcd None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:38.417697 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f58ebb-202a-4211-9020-2ea86735a466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:38.419131 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f58ebb-202a-4211-9020-2ea86735a466 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:38.419780 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f58ebb-202a-4211-9020-2ea86735a466 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:38.420665 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f58ebb-202a-4211-9020-2ea86735a466 admin admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:38.422156 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f58ebb-202a-4211-9020-2ea86735a466 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:15:38.423226 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f58ebb-202a-4211-9020-2ea86735a466 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:38.435790 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f58ebb-202a-4211-9020-2ea86735a466 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:38.500940 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea83be3-38d3-40fd-b7bc-3d343cb7c4bf None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:38.502037 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1023/2042] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:15:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:15:38.512781 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea83be3-38d3-40fd-b7bc-3d343cb7c4bf None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:38.540745 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3d1468-03fd-4516-8011-7ebb689d2aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:38.541881 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3d1468-03fd-4516-8011-7ebb689d2aeb admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:38.542358 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3d1468-03fd-4516-8011-7ebb689d2aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:38.542868 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3d1468-03fd-4516-8011-7ebb689d2aeb admin admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:38.543839 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3d1468-03fd-4516-8011-7ebb689d2aeb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:38.547533 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3d1468-03fd-4516-8011-7ebb689d2aeb admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:38.609613 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1020/2043] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:15:38 2022] POST /identity/v3/projects => generated 375 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 16:15:38.618218 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f58ebb-202a-4211-9020-2ea86735a466 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:38.639558 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f58ebb-202a-4211-9020-2ea86735a466 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:38.748880 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe8ef46-b3db-4d51-8518-3a23cb17cf38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:38.750305 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe8ef46-b3db-4d51-8518-3a23cb17cf38 admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:38.750906 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe8ef46-b3db-4d51-8518-3a23cb17cf38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:38.751302 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe8ef46-b3db-4d51-8518-3a23cb17cf38 admin admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:38.752427 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe8ef46-b3db-4d51-8518-3a23cb17cf38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:38.755875 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe8ef46-b3db-4d51-8518-3a23cb17cf38 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:38.786412 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:15:38.799098 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1024/2044] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:15:38 2022] POST /identity/v3/users => generated 288 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:15:38.799641 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3d1468-03fd-4516-8011-7ebb689d2aeb admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:38.820511 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3d1468-03fd-4516-8011-7ebb689d2aeb admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:38.827964 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ec0042-4c50-4431-ad44-57f6c8ac3a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:38.829019 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec0042-4c50-4431-ad44-57f6c8ac3a57 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:38.829369 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec0042-4c50-4431-ad44-57f6c8ac3a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:38.829843 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec0042-4c50-4431-ad44-57f6c8ac3a57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:38.831218 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ec0042-4c50-4431-ad44-57f6c8ac3a57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:38.834611 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ec0042-4c50-4431-ad44-57f6c8ac3a57 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:38.855184 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1021/2045] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:15:38 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:15:38.857084 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe8ef46-b3db-4d51-8518-3a23cb17cf38 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:38.872947 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe8ef46-b3db-4d51-8518-3a23cb17cf38 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:38.892419 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1787ea2d-b3fa-43d6-b991-58d69254365b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:38.893209 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1787ea2d-b3fa-43d6-b991-58d69254365b admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:38.895081 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1787ea2d-b3fa-43d6-b991-58d69254365b admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:38.895081 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1787ea2d-b3fa-43d6-b991-58d69254365b admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:38.895815 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1787ea2d-b3fa-43d6-b991-58d69254365b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:38.903052 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1787ea2d-b3fa-43d6-b991-58d69254365b admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:38.927525 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1025/2046] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:15:38 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:15:38.929528 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ec0042-4c50-4431-ad44-57f6c8ac3a57 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:38.989204 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ec0042-4c50-4431-ad44-57f6c8ac3a57 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:39.001073 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d831ff4-1070-4fce-9733-2b2bee71596e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:39.002760 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d831ff4-1070-4fce-9733-2b2bee71596e admin admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:39.003512 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d831ff4-1070-4fce-9733-2b2bee71596e admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:39.004873 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d831ff4-1070-4fce-9733-2b2bee71596e admin admin] PATH_INFO: `/v3/projects/9967df7b951c48f08eac6a6b9678ee83/users/d1d16e53ab1748ff9118c6803ec3fe72/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:39.055008 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d831ff4-1070-4fce-9733-2b2bee71596e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9967df7b951c48f08eac6a6b9678ee83, user_id=d1d16e53ab1748ff9118c6803ec3fe72, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:39.058933 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d831ff4-1070-4fce-9733-2b2bee71596e admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:39.108109 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1022/2047] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:15:38 2022] PUT /identity/v3/projects/9967df7b951c48f08eac6a6b9678ee83/users/d1d16e53ab1748ff9118c6803ec3fe72/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:15:39.110036 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d3474e-192b-44f6-87ea-f31e12501e09 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:39.110612 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d3474e-192b-44f6-87ea-f31e12501e09 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:39.111059 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d3474e-192b-44f6-87ea-f31e12501e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:39.172005 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-04d3474e-192b-44f6-87ea-f31e12501e09 None None] MFA Rules not processed for user `d1d16e53ab1748ff9118c6803ec3fe72`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:15:39.174424 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-04d3474e-192b-44f6-87ea-f31e12501e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:15:39.304612 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1026/2048] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:15:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:15:39.310171 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d831ff4-1070-4fce-9733-2b2bee71596e admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:39.384900 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:15:39.402985 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d831ff4-1070-4fce-9733-2b2bee71596e admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:39.412490 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eba99f-627b-4f15-8eda-2bcae3d9aeeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:39.413339 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eba99f-627b-4f15-8eda-2bcae3d9aeeb admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:39.413708 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eba99f-627b-4f15-8eda-2bcae3d9aeeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:39.414151 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eba99f-627b-4f15-8eda-2bcae3d9aeeb admin admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:39.415082 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eba99f-627b-4f15-8eda-2bcae3d9aeeb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:39.417436 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eba99f-627b-4f15-8eda-2bcae3d9aeeb admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:39.462777 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1023/2049] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:15:39 2022] POST /identity/v3/projects => generated 375 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:15:39.464400 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d3474e-192b-44f6-87ea-f31e12501e09 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:39.479203 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d3474e-192b-44f6-87ea-f31e12501e09 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:39.570925 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba6f2cb-f71e-414e-b155-87e6ecece95f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:39.571571 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba6f2cb-f71e-414e-b155-87e6ecece95f admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:39.571858 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba6f2cb-f71e-414e-b155-87e6ecece95f admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:39.572137 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba6f2cb-f71e-414e-b155-87e6ecece95f admin admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:39.572824 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba6f2cb-f71e-414e-b155-87e6ecece95f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:39.574944 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba6f2cb-f71e-414e-b155-87e6ecece95f admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:39.608249 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1027/2050] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:15:39 2022] POST /identity/v3/users => generated 294 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:15:39.613361 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eba99f-627b-4f15-8eda-2bcae3d9aeeb admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:39.625093 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eba99f-627b-4f15-8eda-2bcae3d9aeeb admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:39.630961 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af34d6cd-59a8-4455-bf8f-aa2a1ab4b4de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:39.631616 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af34d6cd-59a8-4455-bf8f-aa2a1ab4b4de admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:39.631898 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af34d6cd-59a8-4455-bf8f-aa2a1ab4b4de admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:39.632185 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af34d6cd-59a8-4455-bf8f-aa2a1ab4b4de admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:39.632844 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af34d6cd-59a8-4455-bf8f-aa2a1ab4b4de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:39.634540 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af34d6cd-59a8-4455-bf8f-aa2a1ab4b4de admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:39.647829 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1024/2051] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:15:39.654484 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba6f2cb-f71e-414e-b155-87e6ecece95f admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:39.675220 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba6f2cb-f71e-414e-b155-87e6ecece95f admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:39.681822 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde4ca77-8b9d-4cc1-83f5-1b4904f4b2ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:39.682930 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4ca77-8b9d-4cc1-83f5-1b4904f4b2ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:39.683258 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4ca77-8b9d-4cc1-83f5-1b4904f4b2ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:39.683673 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4ca77-8b9d-4cc1-83f5-1b4904f4b2ac admin admin] PATH_INFO: `/v3/projects/da67bd522faa45ecbf3a751ad27dd962/users/417aaf555bae4e0f8c7686a694fdc124/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:39.703087 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde4ca77-8b9d-4cc1-83f5-1b4904f4b2ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=da67bd522faa45ecbf3a751ad27dd962, user_id=417aaf555bae4e0f8c7686a694fdc124, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:39.706166 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde4ca77-8b9d-4cc1-83f5-1b4904f4b2ac admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:39.730505 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1028/2052] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:15:39 2022] PUT /identity/v3/projects/da67bd522faa45ecbf3a751ad27dd962/users/417aaf555bae4e0f8c7686a694fdc124/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:15:39.734804 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af34d6cd-59a8-4455-bf8f-aa2a1ab4b4de admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:39.748713 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af34d6cd-59a8-4455-bf8f-aa2a1ab4b4de admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:39.841203 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc847470-c038-4acd-b618-33c5a341419e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:39.841955 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc847470-c038-4acd-b618-33c5a341419e admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:39.842335 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc847470-c038-4acd-b618-33c5a341419e admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:39.842737 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc847470-c038-4acd-b618-33c5a341419e admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:39.843490 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc847470-c038-4acd-b618-33c5a341419e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:39.846022 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc847470-c038-4acd-b618-33c5a341419e admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:39.859024 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1025/2053] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:15:39.863520 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde4ca77-8b9d-4cc1-83f5-1b4904f4b2ac admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:39.874512 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde4ca77-8b9d-4cc1-83f5-1b4904f4b2ac admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:39.880056 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fbb2f6-76b5-4fe7-9753-55040570b28f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:39.880856 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fbb2f6-76b5-4fe7-9753-55040570b28f admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:39.881257 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fbb2f6-76b5-4fe7-9753-55040570b28f admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:39.881692 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fbb2f6-76b5-4fe7-9753-55040570b28f admin admin] PATH_INFO: `/v3/projects/da67bd522faa45ecbf3a751ad27dd962/users/417aaf555bae4e0f8c7686a694fdc124/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:39.884261 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fbb2f6-76b5-4fe7-9753-55040570b28f admin admin] RBAC: Authorizing `identity:create_grant(project_id=da67bd522faa45ecbf3a751ad27dd962, user_id=417aaf555bae4e0f8c7686a694fdc124, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:39.887323 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fbb2f6-76b5-4fe7-9753-55040570b28f admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:39.899790 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1029/2054] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:15:39 2022] PUT /identity/v3/projects/da67bd522faa45ecbf3a751ad27dd962/users/417aaf555bae4e0f8c7686a694fdc124/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 16:15:39.915769 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3954aa-0b46-4550-a473-63987468afa7 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:39.916094 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3954aa-0b46-4550-a473-63987468afa7 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:39.916506 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3954aa-0b46-4550-a473-63987468afa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:39.953977 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-fc3954aa-0b46-4550-a473-63987468afa7 None None] MFA Rules not processed for user `417aaf555bae4e0f8c7686a694fdc124`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:15:39.956152 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-fc3954aa-0b46-4550-a473-63987468afa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:15:40.092548 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1026/2055] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:15:39 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:15:40.108092 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fbb2f6-76b5-4fe7-9753-55040570b28f admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:40.112343 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-b9fbb2f6-76b5-4fe7-9753-55040570b28f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:15:40.173915 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:15:40.252544 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fbb2f6-76b5-4fe7-9753-55040570b28f admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:40.272816 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14652a42-af5b-4251-81cc-a51848391c32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:40.274729 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14652a42-af5b-4251-81cc-a51848391c32 service cinder] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:40.275629 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14652a42-af5b-4251-81cc-a51848391c32 service cinder] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:40.276368 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14652a42-af5b-4251-81cc-a51848391c32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:40.437899 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14652a42-af5b-4251-81cc-a51848391c32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:40.441890 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14652a42-af5b-4251-81cc-a51848391c32 service cinder] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:40.486487 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1030/2056] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:15:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:15:43.427971 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3954aa-0b46-4550-a473-63987468afa7 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:43.510900 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:15:43.530778 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3954aa-0b46-4550-a473-63987468afa7 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:43.536421 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b3b981-d2ba-4b45-98f8-ea1b12844f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:43.536862 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b3b981-d2ba-4b45-98f8-ea1b12844f5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:43.537142 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b3b981-d2ba-4b45-98f8-ea1b12844f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:43.537418 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b3b981-d2ba-4b45-98f8-ea1b12844f5e admin admin] PATH_INFO: `/v3/users/d1d16e53ab1748ff9118c6803ec3fe72` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:43.538679 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b3b981-d2ba-4b45-98f8-ea1b12844f5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1d16e53ab1748ff9118c6803ec3fe72)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:43.541156 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b3b981-d2ba-4b45-98f8-ea1b12844f5e admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:43.618458 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-e1b3b981-d2ba-4b45-98f8-ea1b12844f5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1d16e53ab1748ff9118c6803ec3fe72'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:43.619609 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-e1b3b981-d2ba-4b45-98f8-ea1b12844f5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1d16e53ab1748ff9118c6803ec3fe72'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:43.629534 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-e1b3b981-d2ba-4b45-98f8-ea1b12844f5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1d16e53ab1748ff9118c6803ec3fe72'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:43.644091 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-e1b3b981-d2ba-4b45-98f8-ea1b12844f5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1d16e53ab1748ff9118c6803ec3fe72'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:43.646916 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-e1b3b981-d2ba-4b45-98f8-ea1b12844f5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1d16e53ab1748ff9118c6803ec3fe72'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:43.662432 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1027/2057] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:15:43 2022] DELETE /identity/v3/users/d1d16e53ab1748ff9118c6803ec3fe72 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:15:43.668330 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d463d-33dc-4145-89b3-37cf29a4bea2 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:43.668883 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d463d-33dc-4145-89b3-37cf29a4bea2 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:43.668883 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d463d-33dc-4145-89b3-37cf29a4bea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:43.691196 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-f16d463d-33dc-4145-89b3-37cf29a4bea2 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:15:43.692893 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-f16d463d-33dc-4145-89b3-37cf29a4bea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:15:43.761529 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1031/2058] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:15:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:15:43.774082 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b3b981-d2ba-4b45-98f8-ea1b12844f5e admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:43.775584 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-e1b3b981-d2ba-4b45-98f8-ea1b12844f5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:15:43.868156 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b3b981-d2ba-4b45-98f8-ea1b12844f5e admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:43.881732 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be9da1a-a33b-4e97-b0f6-07b64eeb62fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:43.882627 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be9da1a-a33b-4e97-b0f6-07b64eeb62fe service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:43.882949 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be9da1a-a33b-4e97-b0f6-07b64eeb62fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:43.883281 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be9da1a-a33b-4e97-b0f6-07b64eeb62fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:43.918287 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be9da1a-a33b-4e97-b0f6-07b64eeb62fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:43.921128 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be9da1a-a33b-4e97-b0f6-07b64eeb62fe service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:43.933405 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1028/2059] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:15:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:15:44.451995 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16d463d-33dc-4145-89b3-37cf29a4bea2 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:44.453820 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-f16d463d-33dc-4145-89b3-37cf29a4bea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:15:44.484170 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16d463d-33dc-4145-89b3-37cf29a4bea2 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:44.493884 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6267b0f-1130-430d-ae7c-cff995d3139e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:44.495047 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6267b0f-1130-430d-ae7c-cff995d3139e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:44.495444 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6267b0f-1130-430d-ae7c-cff995d3139e admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:44.495936 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6267b0f-1130-430d-ae7c-cff995d3139e admin admin] PATH_INFO: `/v3/projects/9967df7b951c48f08eac6a6b9678ee83` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:44.498198 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6267b0f-1130-430d-ae7c-cff995d3139e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9967df7b951c48f08eac6a6b9678ee83)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:44.501759 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6267b0f-1130-430d-ae7c-cff995d3139e admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:44.617207 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-e6267b0f-1130-430d-ae7c-cff995d3139e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9967df7b951c48f08eac6a6b9678ee83', 'request_id': 'req-e6267b0f-1130-430d-ae7c-cff995d3139e'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:44.632197 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-e6267b0f-1130-430d-ae7c-cff995d3139e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9967df7b951c48f08eac6a6b9678ee83', 'request_id': 'req-e6267b0f-1130-430d-ae7c-cff995d3139e'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:44.640008 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-e6267b0f-1130-430d-ae7c-cff995d3139e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9967df7b951c48f08eac6a6b9678ee83', 'request_id': 'req-e6267b0f-1130-430d-ae7c-cff995d3139e'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:44.663521 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:15:44.668512 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-e6267b0f-1130-430d-ae7c-cff995d3139e admin admin] The token cache is being invalidate because project 9967df7b951c48f08eac6a6b9678ee83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81618) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:15:44.669388 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-e6267b0f-1130-430d-ae7c-cff995d3139e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:44.670673 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-e6267b0f-1130-430d-ae7c-cff995d3139e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:44.675940 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1032/2060] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:15:44 2022] DELETE /identity/v3/projects/9967df7b951c48f08eac6a6b9678ee83 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 16:15:44.678584 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be9da1a-a33b-4e97-b0f6-07b64eeb62fe service neutron] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:44.693009 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be9da1a-a33b-4e97-b0f6-07b64eeb62fe service neutron] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:44.835224 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385a8700-3a5d-4216-a5e3-be56c8ce7d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:44.836367 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385a8700-3a5d-4216-a5e3-be56c8ce7d1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:44.836881 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385a8700-3a5d-4216-a5e3-be56c8ce7d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:44.837349 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385a8700-3a5d-4216-a5e3-be56c8ce7d1d admin admin] PATH_INFO: `/v3/users/417aaf555bae4e0f8c7686a694fdc124` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:44.839599 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385a8700-3a5d-4216-a5e3-be56c8ce7d1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=417aaf555bae4e0f8c7686a694fdc124)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:44.843499 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385a8700-3a5d-4216-a5e3-be56c8ce7d1d admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:44.872547 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:15:44.950910 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-385a8700-3a5d-4216-a5e3-be56c8ce7d1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '417aaf555bae4e0f8c7686a694fdc124'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:44.952151 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-385a8700-3a5d-4216-a5e3-be56c8ce7d1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '417aaf555bae4e0f8c7686a694fdc124'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:44.964749 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-385a8700-3a5d-4216-a5e3-be56c8ce7d1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '417aaf555bae4e0f8c7686a694fdc124'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:45.004749 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-385a8700-3a5d-4216-a5e3-be56c8ce7d1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '417aaf555bae4e0f8c7686a694fdc124'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:45.005865 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-385a8700-3a5d-4216-a5e3-be56c8ce7d1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '417aaf555bae4e0f8c7686a694fdc124'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:45.025264 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1029/2061] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:15:44 2022] DELETE /identity/v3/users/417aaf555bae4e0f8c7686a694fdc124 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:15:45.416450 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6267b0f-1130-430d-ae7c-cff995d3139e admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:45.419268 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-e6267b0f-1130-430d-ae7c-cff995d3139e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:15:45.464352 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6267b0f-1130-430d-ae7c-cff995d3139e admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:45.583928 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cce137f-f10d-4198-aa74-4fcc88d8d9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:45.585019 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce137f-f10d-4198-aa74-4fcc88d8d9d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:45.585514 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce137f-f10d-4198-aa74-4fcc88d8d9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:45.586017 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce137f-f10d-4198-aa74-4fcc88d8d9d7 admin admin] PATH_INFO: `/v3/projects/da67bd522faa45ecbf3a751ad27dd962` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:45.588198 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cce137f-f10d-4198-aa74-4fcc88d8d9d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da67bd522faa45ecbf3a751ad27dd962)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:45.591926 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cce137f-f10d-4198-aa74-4fcc88d8d9d7 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:45.690207 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-5cce137f-f10d-4198-aa74-4fcc88d8d9d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da67bd522faa45ecbf3a751ad27dd962', 'request_id': 'req-5cce137f-f10d-4198-aa74-4fcc88d8d9d7'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:45.703358 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-5cce137f-f10d-4198-aa74-4fcc88d8d9d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da67bd522faa45ecbf3a751ad27dd962', 'request_id': 'req-5cce137f-f10d-4198-aa74-4fcc88d8d9d7'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:45.715106 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-5cce137f-f10d-4198-aa74-4fcc88d8d9d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da67bd522faa45ecbf3a751ad27dd962', 'request_id': 'req-5cce137f-f10d-4198-aa74-4fcc88d8d9d7'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:45.749218 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-5cce137f-f10d-4198-aa74-4fcc88d8d9d7 admin admin] The token cache is being invalidate because project da67bd522faa45ecbf3a751ad27dd962 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81618) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:15:45.749733 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-5cce137f-f10d-4198-aa74-4fcc88d8d9d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:45.750923 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-5cce137f-f10d-4198-aa74-4fcc88d8d9d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:15:45.755280 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1033/2062] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:15:45 2022] DELETE /identity/v3/projects/da67bd522faa45ecbf3a751ad27dd962 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:15:45.762009 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e8543d-21cd-4612-a4ef-b4a28f5276b8 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:45.763027 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e8543d-21cd-4612-a4ef-b4a28f5276b8 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:45.763580 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e8543d-21cd-4612-a4ef-b4a28f5276b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:45.799626 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-46e8543d-21cd-4612-a4ef-b4a28f5276b8 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:15:45.802727 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-46e8543d-21cd-4612-a4ef-b4a28f5276b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:15:45.875325 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1030/2063] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:15:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:15:45.884401 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dd5381-fdec-4993-bc59-8e14ab22bf5d None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:45.884678 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dd5381-fdec-4993-bc59-8e14ab22bf5d None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:45.884937 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dd5381-fdec-4993-bc59-8e14ab22bf5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:45.909712 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-80dd5381-fdec-4993-bc59-8e14ab22bf5d None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:15:45.911874 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-80dd5381-fdec-4993-bc59-8e14ab22bf5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:15:45.926872 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1034/2064] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:15:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:15:45.932571 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e8543d-21cd-4612-a4ef-b4a28f5276b8 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:45.947268 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e8543d-21cd-4612-a4ef-b4a28f5276b8 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:45.953889 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd82c386-b63d-474a-a0c6-c6aa16e50702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:45.954597 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd82c386-b63d-474a-a0c6-c6aa16e50702 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:45.954877 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd82c386-b63d-474a-a0c6-c6aa16e50702 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:45.955155 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd82c386-b63d-474a-a0c6-c6aa16e50702 admin admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:45.955978 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd82c386-b63d-474a-a0c6-c6aa16e50702 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:15:45.956285 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd82c386-b63d-474a-a0c6-c6aa16e50702 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:45.957909 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd82c386-b63d-474a-a0c6-c6aa16e50702 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:45.971251 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1031/2065] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:15:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:15:45.976571 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dd5381-fdec-4993-bc59-8e14ab22bf5d None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:45.988839 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dd5381-fdec-4993-bc59-8e14ab22bf5d None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:45.994139 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbadbc13-1c79-49cd-bbbe-5138dee8e1b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:45.994905 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbadbc13-1c79-49cd-bbbe-5138dee8e1b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:45.995320 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbadbc13-1c79-49cd-bbbe-5138dee8e1b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:45.995678 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbadbc13-1c79-49cd-bbbe-5138dee8e1b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:45.996508 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbadbc13-1c79-49cd-bbbe-5138dee8e1b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:45.998673 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbadbc13-1c79-49cd-bbbe-5138dee8e1b5 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:46.033369 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1035/2066] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:15:45 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:15:46.034917 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd82c386-b63d-474a-a0c6-c6aa16e50702 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:46.059633 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd82c386-b63d-474a-a0c6-c6aa16e50702 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:46.162811 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a42229-6b62-4481-9b8e-d4ca9b11553b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:46.163798 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a42229-6b62-4481-9b8e-d4ca9b11553b admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:46.164392 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a42229-6b62-4481-9b8e-d4ca9b11553b admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:46.165003 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a42229-6b62-4481-9b8e-d4ca9b11553b admin admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:46.166001 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a42229-6b62-4481-9b8e-d4ca9b11553b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:46.169827 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a42229-6b62-4481-9b8e-d4ca9b11553b admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:46.203026 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1032/2067] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:15:46 2022] POST /identity/v3/users => generated 286 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 16:15:46.209776 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbadbc13-1c79-49cd-bbbe-5138dee8e1b5 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:46.221908 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbadbc13-1c79-49cd-bbbe-5138dee8e1b5 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:46.229250 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326b1be5-0e52-4ed5-8bb9-0f8a957d4fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:46.230469 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b1be5-0e52-4ed5-8bb9-0f8a957d4fac admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:46.231089 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b1be5-0e52-4ed5-8bb9-0f8a957d4fac admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:46.231740 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b1be5-0e52-4ed5-8bb9-0f8a957d4fac admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:46.233040 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326b1be5-0e52-4ed5-8bb9-0f8a957d4fac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:46.237475 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326b1be5-0e52-4ed5-8bb9-0f8a957d4fac admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:46.263031 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1036/2068] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:15:46 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 16:15:46.266956 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a42229-6b62-4481-9b8e-d4ca9b11553b admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:46.281760 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:15:46.287373 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a42229-6b62-4481-9b8e-d4ca9b11553b admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:46.294823 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec77fb35-ee74-455a-8ff2-74e276dfcd57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:46.296401 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec77fb35-ee74-455a-8ff2-74e276dfcd57 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:46.297000 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec77fb35-ee74-455a-8ff2-74e276dfcd57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:46.297303 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec77fb35-ee74-455a-8ff2-74e276dfcd57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:46.298065 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec77fb35-ee74-455a-8ff2-74e276dfcd57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:46.300020 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec77fb35-ee74-455a-8ff2-74e276dfcd57 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:46.315126 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1033/2069] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:15:46 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:15:46.320135 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326b1be5-0e52-4ed5-8bb9-0f8a957d4fac admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:46.336190 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326b1be5-0e52-4ed5-8bb9-0f8a957d4fac admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:46.341167 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ddf84a-dc17-4ae9-b63b-d68ba5c50f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:46.341886 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ddf84a-dc17-4ae9-b63b-d68ba5c50f4c admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:46.342159 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ddf84a-dc17-4ae9-b63b-d68ba5c50f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:46.342475 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ddf84a-dc17-4ae9-b63b-d68ba5c50f4c admin admin] PATH_INFO: `/v3/projects/56d3906965284a8d92ab83e811299d4f/users/11794279b67342e78ff86b6b9d34bf68/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:46.359026 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ddf84a-dc17-4ae9-b63b-d68ba5c50f4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=56d3906965284a8d92ab83e811299d4f, user_id=11794279b67342e78ff86b6b9d34bf68, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:46.363944 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ddf84a-dc17-4ae9-b63b-d68ba5c50f4c admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:46.381176 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1037/2070] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:15:46 2022] PUT /identity/v3/projects/56d3906965284a8d92ab83e811299d4f/users/11794279b67342e78ff86b6b9d34bf68/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:15:46.392853 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0d69c3-134c-4316-ae30-1cf4a6d05b9a None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:46.393190 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0d69c3-134c-4316-ae30-1cf4a6d05b9a None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:46.393521 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0d69c3-134c-4316-ae30-1cf4a6d05b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:46.430394 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-da0d69c3-134c-4316-ae30-1cf4a6d05b9a None None] MFA Rules not processed for user `11794279b67342e78ff86b6b9d34bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:15:46.431638 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-da0d69c3-134c-4316-ae30-1cf4a6d05b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:15:46.505605 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1034/2071] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:15:46 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:15:46.519299 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ddf84a-dc17-4ae9-b63b-d68ba5c50f4c admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:15:46.521008 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-47ddf84a-dc17-4ae9-b63b-d68ba5c50f4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:15:46.573483 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:15:46.611095 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ddf84a-dc17-4ae9-b63b-d68ba5c50f4c admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:15:46.616577 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a5d2a6-f52f-4d4a-9f3a-45d4b8cceb02 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5632ade0cc6f4ae28f72160de68486cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:15:46.617333 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a5d2a6-f52f-4d4a-9f3a-45d4b8cceb02 service glance] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:15:46.617718 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a5d2a6-f52f-4d4a-9f3a-45d4b8cceb02 service glance] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:15:46.618092 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a5d2a6-f52f-4d4a-9f3a-45d4b8cceb02 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:15:46.629613 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a5d2a6-f52f-4d4a-9f3a-45d4b8cceb02 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:15:46.631198 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a5d2a6-f52f-4d4a-9f3a-45d4b8cceb02 service glance] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:15:46.643005 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1038/2072] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:15:46 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 16:15:56.501461 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=35 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:15:56.636899 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=14 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:16:01.207125 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcefd0cb-1a21-4083-9f26-46d2d88faf9d None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:01.208744 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcefd0cb-1a21-4083-9f26-46d2d88faf9d None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:01.209425 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcefd0cb-1a21-4083-9f26-46d2d88faf9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:01.290265 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-bcefd0cb-1a21-4083-9f26-46d2d88faf9d None None] MFA Rules not processed for user `7222bd3b313b47e5a4fa0ebc9ff0cdd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:16:01.292719 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-bcefd0cb-1a21-4083-9f26-46d2d88faf9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:16:01.399103 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1035/2073] 10.222.0.57 () {58 vars in 1009 bytes} [Mon Sep 19 16:16:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:16:01.475722 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a5d2a6-f52f-4d4a-9f3a-45d4b8cceb02 service glance] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:01.477432 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-b5a5d2a6-f52f-4d4a-9f3a-45d4b8cceb02 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:16:01.568883 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a5d2a6-f52f-4d4a-9f3a-45d4b8cceb02 service glance] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:01.574132 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba747c3-66df-4ecd-9039-8aa6d6e4a75f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '884a0524aca744b99c7496c11c3291ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:01.574802 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba747c3-66df-4ecd-9039-8aa6d6e4a75f service swift] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:01.575079 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba747c3-66df-4ecd-9039-8aa6d6e4a75f service swift] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:01.575349 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba747c3-66df-4ecd-9039-8aa6d6e4a75f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:01.586281 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba747c3-66df-4ecd-9039-8aa6d6e4a75f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:01.588311 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba747c3-66df-4ecd-9039-8aa6d6e4a75f service swift] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:01.599412 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1039/2074] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Sep 19 16:16:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 19 16:16:05.551796 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e8626-1923-4fa3-97a1-57196fcc6e17 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:05.552689 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e8626-1923-4fa3-97a1-57196fcc6e17 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:05.553713 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e8626-1923-4fa3-97a1-57196fcc6e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:05.583609 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-f66e8626-1923-4fa3-97a1-57196fcc6e17 None None] MFA Rules not processed for user `9fd6c22a61be43658ae87052b867e384`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:16:05.584914 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-f66e8626-1923-4fa3-97a1-57196fcc6e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:16:05.686429 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1036/2075] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:16:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:16:11.594804 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=32 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:16:12.425785 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a7ee58-f964-4bb8-9451-e8f978a409ca None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:12.426225 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a7ee58-f964-4bb8-9451-e8f978a409ca None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:12.426804 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a7ee58-f964-4bb8-9451-e8f978a409ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:12.451006 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-c5a7ee58-f964-4bb8-9451-e8f978a409ca None None] MFA Rules not processed for user `fd36089b3c2141218660ab2458376798`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:16:12.452297 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-c5a7ee58-f964-4bb8-9451-e8f978a409ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:16:12.558545 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1040/2076] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:16:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:16:12.902159 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e6c73-6db7-4705-be0a-608403ae63d9 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:12.905833 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e6c73-6db7-4705-be0a-608403ae63d9 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:12.906698 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e6c73-6db7-4705-be0a-608403ae63d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:12.982157 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-eb3e6c73-6db7-4705-be0a-608403ae63d9 None None] MFA Rules not processed for user `8cd9c727c0b444cb83497ea4f78e24d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:16:12.984117 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-eb3e6c73-6db7-4705-be0a-608403ae63d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:16:13.164409 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1037/2077] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:16:12 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:16:18.740993 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a7ee58-f964-4bb8-9451-e8f978a409ca None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:18.749488 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-c5a7ee58-f964-4bb8-9451-e8f978a409ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:16:19.143785 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a7ee58-f964-4bb8-9451-e8f978a409ca None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:19.201584 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944eb326-85d1-4adc-8034-822d497d1585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:19.202729 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944eb326-85d1-4adc-8034-822d497d1585 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:19.205934 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944eb326-85d1-4adc-8034-822d497d1585 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:19.206548 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944eb326-85d1-4adc-8034-822d497d1585 admin admin] PATH_INFO: `/v3/users/d3fc9058ff4f484fa592228411954b71` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:19.216878 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944eb326-85d1-4adc-8034-822d497d1585 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3fc9058ff4f484fa592228411954b71)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:19.239420 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944eb326-85d1-4adc-8034-822d497d1585 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:19.555926 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-944eb326-85d1-4adc-8034-822d497d1585 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3fc9058ff4f484fa592228411954b71'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:19.573628 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-944eb326-85d1-4adc-8034-822d497d1585 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3fc9058ff4f484fa592228411954b71'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:19.600348 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-944eb326-85d1-4adc-8034-822d497d1585 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3fc9058ff4f484fa592228411954b71'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:19.616907 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-944eb326-85d1-4adc-8034-822d497d1585 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3fc9058ff4f484fa592228411954b71'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:19.617558 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-944eb326-85d1-4adc-8034-822d497d1585 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3fc9058ff4f484fa592228411954b71'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:19.636189 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1041/2078] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:16:18 2022] DELETE /identity/v3/users/d3fc9058ff4f484fa592228411954b71 => generated 0 bytes in 897 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:16:19.642108 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8c8502-89dc-44e0-9e98-397a64b94d5a None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:19.643139 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8c8502-89dc-44e0-9e98-397a64b94d5a None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:19.643959 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8c8502-89dc-44e0-9e98-397a64b94d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:19.705977 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-0b8c8502-89dc-44e0-9e98-397a64b94d5a None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:16:19.714279 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-0b8c8502-89dc-44e0-9e98-397a64b94d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:16:19.970193 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:16:20.013625 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1038/2079] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:16:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:16:20.032680 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944eb326-85d1-4adc-8034-822d497d1585 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:20.036341 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-944eb326-85d1-4adc-8034-822d497d1585 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:16:20.191988 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:16:20.385118 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944eb326-85d1-4adc-8034-822d497d1585 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:20.392434 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1fffaa-e7e8-4eb6-b95d-79a799b29890 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:20.393701 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1fffaa-e7e8-4eb6-b95d-79a799b29890 service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:20.394220 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1fffaa-e7e8-4eb6-b95d-79a799b29890 service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:20.394766 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1fffaa-e7e8-4eb6-b95d-79a799b29890 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:20.428779 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1fffaa-e7e8-4eb6-b95d-79a799b29890 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:20.431996 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1fffaa-e7e8-4eb6-b95d-79a799b29890 service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:20.499373 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1042/2080] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:16:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 16:16:22.740264 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8c8502-89dc-44e0-9e98-397a64b94d5a None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:22.743996 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-0b8c8502-89dc-44e0-9e98-397a64b94d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:16:22.806690 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8c8502-89dc-44e0-9e98-397a64b94d5a None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:22.843735 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94122e9-ebd5-41ef-8e15-8e7bd13e68aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:22.844789 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94122e9-ebd5-41ef-8e15-8e7bd13e68aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:22.845183 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94122e9-ebd5-41ef-8e15-8e7bd13e68aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:22.845760 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94122e9-ebd5-41ef-8e15-8e7bd13e68aa admin admin] PATH_INFO: `/v3/projects/48dbc60f791341f9a8d3e68ea865ce27` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:22.847997 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94122e9-ebd5-41ef-8e15-8e7bd13e68aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=48dbc60f791341f9a8d3e68ea865ce27)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:22.851276 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94122e9-ebd5-41ef-8e15-8e7bd13e68aa admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:22.964325 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-a94122e9-ebd5-41ef-8e15-8e7bd13e68aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48dbc60f791341f9a8d3e68ea865ce27', 'request_id': 'req-a94122e9-ebd5-41ef-8e15-8e7bd13e68aa'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:22.981669 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-a94122e9-ebd5-41ef-8e15-8e7bd13e68aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48dbc60f791341f9a8d3e68ea865ce27', 'request_id': 'req-a94122e9-ebd5-41ef-8e15-8e7bd13e68aa'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:23.000133 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-a94122e9-ebd5-41ef-8e15-8e7bd13e68aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48dbc60f791341f9a8d3e68ea865ce27', 'request_id': 'req-a94122e9-ebd5-41ef-8e15-8e7bd13e68aa'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:23.030535 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-a94122e9-ebd5-41ef-8e15-8e7bd13e68aa admin admin] The token cache is being invalidate because project 48dbc60f791341f9a8d3e68ea865ce27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81617) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:16:23.030834 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-a94122e9-ebd5-41ef-8e15-8e7bd13e68aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:23.031934 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-a94122e9-ebd5-41ef-8e15-8e7bd13e68aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:23.034646 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1039/2081] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:16:22 2022] DELETE /identity/v3/projects/48dbc60f791341f9a8d3e68ea865ce27 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:16:23.040187 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1fffaa-e7e8-4eb6-b95d-79a799b29890 service neutron] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:23.077323 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1fffaa-e7e8-4eb6-b95d-79a799b29890 service neutron] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:23.286693 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae567e5a-c01f-4eb3-9a61-bc7835a26c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:23.287858 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae567e5a-c01f-4eb3-9a61-bc7835a26c49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:23.288342 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae567e5a-c01f-4eb3-9a61-bc7835a26c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:23.288822 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae567e5a-c01f-4eb3-9a61-bc7835a26c49 admin admin] PATH_INFO: `/v3/users/e1ffcd8687cc451fbb84b849e592693e` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:23.291189 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae567e5a-c01f-4eb3-9a61-bc7835a26c49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1ffcd8687cc451fbb84b849e592693e)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:23.295747 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae567e5a-c01f-4eb3-9a61-bc7835a26c49 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:23.456479 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-ae567e5a-c01f-4eb3-9a61-bc7835a26c49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1ffcd8687cc451fbb84b849e592693e'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:23.483129 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-ae567e5a-c01f-4eb3-9a61-bc7835a26c49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1ffcd8687cc451fbb84b849e592693e'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:23.548734 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-ae567e5a-c01f-4eb3-9a61-bc7835a26c49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1ffcd8687cc451fbb84b849e592693e'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:23.590807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-ae567e5a-c01f-4eb3-9a61-bc7835a26c49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1ffcd8687cc451fbb84b849e592693e'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:23.592349 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-ae567e5a-c01f-4eb3-9a61-bc7835a26c49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1ffcd8687cc451fbb84b849e592693e'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:23.626183 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1043/2082] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:16:23 2022] DELETE /identity/v3/users/e1ffcd8687cc451fbb84b849e592693e => generated 0 bytes in 581 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:16:25.237595 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94122e9-ebd5-41ef-8e15-8e7bd13e68aa admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:25.239675 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-a94122e9-ebd5-41ef-8e15-8e7bd13e68aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:16:25.276856 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94122e9-ebd5-41ef-8e15-8e7bd13e68aa admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:25.357785 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e166b2-6f91-493d-b51f-16e49af8863a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:25.358475 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e166b2-6f91-493d-b51f-16e49af8863a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:25.358680 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e166b2-6f91-493d-b51f-16e49af8863a admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:25.358991 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e166b2-6f91-493d-b51f-16e49af8863a admin admin] PATH_INFO: `/v3/projects/90a66807e477439a9b1c3df49ee7d5ff` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:25.361000 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e166b2-6f91-493d-b51f-16e49af8863a admin admin] RBAC: Authorizing `identity:delete_project(project_id=90a66807e477439a9b1c3df49ee7d5ff)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:25.363094 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e166b2-6f91-493d-b51f-16e49af8863a admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:25.436170 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-d9e166b2-6f91-493d-b51f-16e49af8863a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90a66807e477439a9b1c3df49ee7d5ff', 'request_id': 'req-d9e166b2-6f91-493d-b51f-16e49af8863a'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:25.446846 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-d9e166b2-6f91-493d-b51f-16e49af8863a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90a66807e477439a9b1c3df49ee7d5ff', 'request_id': 'req-d9e166b2-6f91-493d-b51f-16e49af8863a'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:25.454572 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-d9e166b2-6f91-493d-b51f-16e49af8863a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90a66807e477439a9b1c3df49ee7d5ff', 'request_id': 'req-d9e166b2-6f91-493d-b51f-16e49af8863a'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:25.494965 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-d9e166b2-6f91-493d-b51f-16e49af8863a admin admin] The token cache is being invalidate because project 90a66807e477439a9b1c3df49ee7d5ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81617) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:16:25.495489 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-d9e166b2-6f91-493d-b51f-16e49af8863a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:25.497681 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-d9e166b2-6f91-493d-b51f-16e49af8863a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:25.508024 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1040/2083] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:16:25 2022] DELETE /identity/v3/projects/90a66807e477439a9b1c3df49ee7d5ff => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:16:25.509492 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a5a2f2-f03c-43de-8ded-0ae19349ad51 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:25.510267 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a5a2f2-f03c-43de-8ded-0ae19349ad51 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:25.510267 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a5a2f2-f03c-43de-8ded-0ae19349ad51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:25.539129 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-a7a5a2f2-f03c-43de-8ded-0ae19349ad51 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:16:25.540454 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-a7a5a2f2-f03c-43de-8ded-0ae19349ad51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:16:25.618941 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:16:25.648933 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1044/2084] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:16:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:16:25.657893 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c27eb-5af7-4b8a-9e80-e4ccb60342a5 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:25.658725 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c27eb-5af7-4b8a-9e80-e4ccb60342a5 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:25.659009 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c27eb-5af7-4b8a-9e80-e4ccb60342a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:25.726982 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-eb0c27eb-5af7-4b8a-9e80-e4ccb60342a5 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:16:25.728419 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-eb0c27eb-5af7-4b8a-9e80-e4ccb60342a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:16:25.752451 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1041/2085] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:16:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:16:25.754815 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a5a2f2-f03c-43de-8ded-0ae19349ad51 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:25.812963 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a5a2f2-f03c-43de-8ded-0ae19349ad51 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:25.826120 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1ddda1-2c0c-4607-ac4b-1334a838d6e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:25.826460 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ddda1-2c0c-4607-ac4b-1334a838d6e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:25.826873 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ddda1-2c0c-4607-ac4b-1334a838d6e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:25.827395 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ddda1-2c0c-4607-ac4b-1334a838d6e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:25.829746 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1ddda1-2c0c-4607-ac4b-1334a838d6e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:16:25.829746 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1ddda1-2c0c-4607-ac4b-1334a838d6e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:25.833427 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1ddda1-2c0c-4607-ac4b-1334a838d6e3 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:25.864564 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1045/2086] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:16:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:16:25.869410 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c27eb-5af7-4b8a-9e80-e4ccb60342a5 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:25.879883 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:16:25.901713 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c27eb-5af7-4b8a-9e80-e4ccb60342a5 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:25.911838 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e297426e-5c0c-445e-8cb6-597854dcbd41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:25.913633 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297426e-5c0c-445e-8cb6-597854dcbd41 admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:25.914612 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297426e-5c0c-445e-8cb6-597854dcbd41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:25.915435 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297426e-5c0c-445e-8cb6-597854dcbd41 admin admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:25.916844 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e297426e-5c0c-445e-8cb6-597854dcbd41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:25.934712 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e297426e-5c0c-445e-8cb6-597854dcbd41 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:26.019810 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1042/2087] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:16:25 2022] POST /identity/v3/projects => generated 389 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:16:26.022853 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1ddda1-2c0c-4607-ac4b-1334a838d6e3 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:26.057748 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1ddda1-2c0c-4607-ac4b-1334a838d6e3 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:26.245287 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435748f6-b665-4392-a985-d9e325ac4954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:26.246297 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435748f6-b665-4392-a985-d9e325ac4954 admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:26.247283 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435748f6-b665-4392-a985-d9e325ac4954 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:26.253114 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435748f6-b665-4392-a985-d9e325ac4954 admin admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:26.255434 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435748f6-b665-4392-a985-d9e325ac4954 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:26.258663 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435748f6-b665-4392-a985-d9e325ac4954 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:26.355778 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1046/2088] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:16:26 2022] POST /identity/v3/users => generated 295 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:16:26.357692 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e297426e-5c0c-445e-8cb6-597854dcbd41 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:26.387426 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e297426e-5c0c-445e-8cb6-597854dcbd41 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:26.405624 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2af23c1-6ae1-40a7-9e8d-1f77d96dc4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:26.406278 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af23c1-6ae1-40a7-9e8d-1f77d96dc4ea admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:26.406751 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af23c1-6ae1-40a7-9e8d-1f77d96dc4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:26.407151 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af23c1-6ae1-40a7-9e8d-1f77d96dc4ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:26.407987 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2af23c1-6ae1-40a7-9e8d-1f77d96dc4ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:26.415076 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2af23c1-6ae1-40a7-9e8d-1f77d96dc4ea admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:26.441146 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1043/2089] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:16:26 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:16:26.445785 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435748f6-b665-4392-a985-d9e325ac4954 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:26.480192 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435748f6-b665-4392-a985-d9e325ac4954 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:26.491157 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b41ef4-1975-4e76-af86-801116073dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:26.492393 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b41ef4-1975-4e76-af86-801116073dda admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:26.492880 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b41ef4-1975-4e76-af86-801116073dda admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:26.493362 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b41ef4-1975-4e76-af86-801116073dda admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:26.494637 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b41ef4-1975-4e76-af86-801116073dda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:26.497758 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b41ef4-1975-4e76-af86-801116073dda admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:26.532013 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1047/2090] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:16:26 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:16:26.536628 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2af23c1-6ae1-40a7-9e8d-1f77d96dc4ea admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:26.550318 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2af23c1-6ae1-40a7-9e8d-1f77d96dc4ea admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:26.566449 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8c7937-4cea-4e8e-ba4a-2785a31c97e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:26.567218 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8c7937-4cea-4e8e-ba4a-2785a31c97e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:26.567642 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8c7937-4cea-4e8e-ba4a-2785a31c97e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:26.567807 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8c7937-4cea-4e8e-ba4a-2785a31c97e4 admin admin] PATH_INFO: `/v3/projects/4f35b48adcff4e0cbc3aa3c7e88acf27/users/3a682d75ea174d20bb3b5751bd3582e3/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:26.609234 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8c7937-4cea-4e8e-ba4a-2785a31c97e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f35b48adcff4e0cbc3aa3c7e88acf27, user_id=3a682d75ea174d20bb3b5751bd3582e3, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:26.612244 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8c7937-4cea-4e8e-ba4a-2785a31c97e4 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:26.676142 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1044/2091] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:16:26 2022] PUT /identity/v3/projects/4f35b48adcff4e0cbc3aa3c7e88acf27/users/3a682d75ea174d20bb3b5751bd3582e3/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:16:26.691139 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe20a1f-7081-449f-aa3c-619a665e90f5 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:26.691782 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe20a1f-7081-449f-aa3c-619a665e90f5 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:26.692510 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe20a1f-7081-449f-aa3c-619a665e90f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:26.878911 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-fbe20a1f-7081-449f-aa3c-619a665e90f5 None None] MFA Rules not processed for user `3a682d75ea174d20bb3b5751bd3582e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:16:26.886098 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-fbe20a1f-7081-449f-aa3c-619a665e90f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:16:27.176276 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1048/2092] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:16:26 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 485 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:16:27.178737 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8c7937-4cea-4e8e-ba4a-2785a31c97e4 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:27.490813 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8c7937-4cea-4e8e-ba4a-2785a31c97e4 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:27.501140 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97667f89-139d-46ea-bbfc-e2e90160147a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:27.503267 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97667f89-139d-46ea-bbfc-e2e90160147a admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:27.504182 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97667f89-139d-46ea-bbfc-e2e90160147a admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:27.504871 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97667f89-139d-46ea-bbfc-e2e90160147a admin admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:27.506665 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97667f89-139d-46ea-bbfc-e2e90160147a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:27.510876 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97667f89-139d-46ea-bbfc-e2e90160147a admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:27.595486 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1045/2093] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:16:27 2022] POST /identity/v3/projects => generated 389 bytes in 418 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:16:27.601130 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe20a1f-7081-449f-aa3c-619a665e90f5 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:27.646507 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe20a1f-7081-449f-aa3c-619a665e90f5 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:27.782548 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:16:27.823514 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd50f44a-e4fb-4227-83c0-c9be70bc1c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:27.823859 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd50f44a-e4fb-4227-83c0-c9be70bc1c60 admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:27.824134 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd50f44a-e4fb-4227-83c0-c9be70bc1c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:27.824454 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd50f44a-e4fb-4227-83c0-c9be70bc1c60 admin admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:27.825125 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd50f44a-e4fb-4227-83c0-c9be70bc1c60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:27.829585 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd50f44a-e4fb-4227-83c0-c9be70bc1c60 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:27.884700 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1049/2094] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:16:27 2022] POST /identity/v3/users => generated 301 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:16:27.889704 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97667f89-139d-46ea-bbfc-e2e90160147a admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:27.931005 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97667f89-139d-46ea-bbfc-e2e90160147a admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:27.959264 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e36eae-98d4-4840-98cb-d62e213c5762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:27.963777 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e36eae-98d4-4840-98cb-d62e213c5762 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:27.964426 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e36eae-98d4-4840-98cb-d62e213c5762 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:27.964710 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e36eae-98d4-4840-98cb-d62e213c5762 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:27.965700 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e36eae-98d4-4840-98cb-d62e213c5762 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:27.967977 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e36eae-98d4-4840-98cb-d62e213c5762 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:27.995575 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1046/2095] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:16:27 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:16:27.999971 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd50f44a-e4fb-4227-83c0-c9be70bc1c60 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:28.013288 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd50f44a-e4fb-4227-83c0-c9be70bc1c60 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:28.018929 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e9b5ef-1432-4e6d-ad37-afba439dd252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:28.019904 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e9b5ef-1432-4e6d-ad37-afba439dd252 admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:28.020201 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e9b5ef-1432-4e6d-ad37-afba439dd252 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:28.020528 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e9b5ef-1432-4e6d-ad37-afba439dd252 admin admin] PATH_INFO: `/v3/projects/5e9c92e7353d443cb90bc5e22d4ddad9/users/b29fd62d4e8c484bb2bf640ba44c9280/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:28.039012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e9b5ef-1432-4e6d-ad37-afba439dd252 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e9c92e7353d443cb90bc5e22d4ddad9, user_id=b29fd62d4e8c484bb2bf640ba44c9280, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:28.042001 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e9b5ef-1432-4e6d-ad37-afba439dd252 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:28.062507 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1050/2096] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:16:27 2022] PUT /identity/v3/projects/5e9c92e7353d443cb90bc5e22d4ddad9/users/b29fd62d4e8c484bb2bf640ba44c9280/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:16:28.066779 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e36eae-98d4-4840-98cb-d62e213c5762 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:28.078079 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e36eae-98d4-4840-98cb-d62e213c5762 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:28.168207 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f52925-c5b6-4555-b932-08e0a633021f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:28.168842 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f52925-c5b6-4555-b932-08e0a633021f admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:28.169244 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f52925-c5b6-4555-b932-08e0a633021f admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:28.169528 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f52925-c5b6-4555-b932-08e0a633021f admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:28.170951 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f52925-c5b6-4555-b932-08e0a633021f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:28.172674 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f52925-c5b6-4555-b932-08e0a633021f admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:28.191564 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1047/2097] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:16:28 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:16:28.192731 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e9b5ef-1432-4e6d-ad37-afba439dd252 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:28.214007 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e9b5ef-1432-4e6d-ad37-afba439dd252 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:28.226329 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56e4c8f-0913-4bd5-9a5d-cdaf10bee131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:28.227242 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e4c8f-0913-4bd5-9a5d-cdaf10bee131 admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:28.227619 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e4c8f-0913-4bd5-9a5d-cdaf10bee131 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:28.227991 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e4c8f-0913-4bd5-9a5d-cdaf10bee131 admin admin] PATH_INFO: `/v3/projects/5e9c92e7353d443cb90bc5e22d4ddad9/users/b29fd62d4e8c484bb2bf640ba44c9280/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:28.230560 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56e4c8f-0913-4bd5-9a5d-cdaf10bee131 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e9c92e7353d443cb90bc5e22d4ddad9, user_id=b29fd62d4e8c484bb2bf640ba44c9280, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:28.233806 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56e4c8f-0913-4bd5-9a5d-cdaf10bee131 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:28.280825 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1051/2098] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:16:28 2022] PUT /identity/v3/projects/5e9c92e7353d443cb90bc5e22d4ddad9/users/b29fd62d4e8c484bb2bf640ba44c9280/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 16:16:28.284814 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44bfa05-bd09-41b7-bf72-936165abcc91 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:28.285382 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44bfa05-bd09-41b7-bf72-936165abcc91 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:28.285851 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44bfa05-bd09-41b7-bf72-936165abcc91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:28.350768 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-a44bfa05-bd09-41b7-bf72-936165abcc91 None None] MFA Rules not processed for user `b29fd62d4e8c484bb2bf640ba44c9280`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:16:28.352368 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-a44bfa05-bd09-41b7-bf72-936165abcc91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:16:28.363104 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:16:28.530414 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1048/2099] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:16:28 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:16:28.540489 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56e4c8f-0913-4bd5-9a5d-cdaf10bee131 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:28.542701 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-e56e4c8f-0913-4bd5-9a5d-cdaf10bee131 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:16:28.671400 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56e4c8f-0913-4bd5-9a5d-cdaf10bee131 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:28.685726 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5de2ae-65de-4516-b7bf-529dafd9296a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:28.691028 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5de2ae-65de-4516-b7bf-529dafd9296a service cinder] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:28.691589 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5de2ae-65de-4516-b7bf-529dafd9296a service cinder] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:28.691962 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5de2ae-65de-4516-b7bf-529dafd9296a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:28.704743 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5de2ae-65de-4516-b7bf-529dafd9296a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:28.707036 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5de2ae-65de-4516-b7bf-529dafd9296a service cinder] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:28.722782 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1052/2100] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:16:28 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:16:28.969593 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44bfa05-bd09-41b7-bf72-936165abcc91 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:29.088942 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44bfa05-bd09-41b7-bf72-936165abcc91 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:29.104864 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d87badd-c4c8-4c3c-b1a2-60c0e1ce4539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:29.105808 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87badd-c4c8-4c3c-b1a2-60c0e1ce4539 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:29.106274 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87badd-c4c8-4c3c-b1a2-60c0e1ce4539 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:29.106759 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87badd-c4c8-4c3c-b1a2-60c0e1ce4539 admin admin] PATH_INFO: `/v3/users/3a682d75ea174d20bb3b5751bd3582e3` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:29.108225 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d87badd-c4c8-4c3c-b1a2-60c0e1ce4539 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a682d75ea174d20bb3b5751bd3582e3)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:29.110703 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d87badd-c4c8-4c3c-b1a2-60c0e1ce4539 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:29.196090 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-4d87badd-c4c8-4c3c-b1a2-60c0e1ce4539 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a682d75ea174d20bb3b5751bd3582e3'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:29.199006 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-4d87badd-c4c8-4c3c-b1a2-60c0e1ce4539 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a682d75ea174d20bb3b5751bd3582e3'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:29.209257 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-4d87badd-c4c8-4c3c-b1a2-60c0e1ce4539 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a682d75ea174d20bb3b5751bd3582e3'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:29.232575 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-4d87badd-c4c8-4c3c-b1a2-60c0e1ce4539 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a682d75ea174d20bb3b5751bd3582e3'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:29.233779 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-4d87badd-c4c8-4c3c-b1a2-60c0e1ce4539 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a682d75ea174d20bb3b5751bd3582e3'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:29.256722 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1049/2101] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:16:28 2022] DELETE /identity/v3/users/3a682d75ea174d20bb3b5751bd3582e3 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:16:29.264164 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242a8509-42bc-4108-bc75-665810e86ce3 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:29.264572 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242a8509-42bc-4108-bc75-665810e86ce3 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:29.264971 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242a8509-42bc-4108-bc75-665810e86ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:29.331494 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-242a8509-42bc-4108-bc75-665810e86ce3 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:16:29.334422 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-242a8509-42bc-4108-bc75-665810e86ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:16:29.466528 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1053/2102] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:16:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:16:29.481307 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d87badd-c4c8-4c3c-b1a2-60c0e1ce4539 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:29.483704 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-4d87badd-c4c8-4c3c-b1a2-60c0e1ce4539 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:16:29.526725 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:16:29.711428 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d87badd-c4c8-4c3c-b1a2-60c0e1ce4539 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:29.726763 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45335d9-3977-4144-b31c-a7c0555a0781 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:29.728023 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45335d9-3977-4144-b31c-a7c0555a0781 service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:29.728709 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45335d9-3977-4144-b31c-a7c0555a0781 service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:29.730526 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45335d9-3977-4144-b31c-a7c0555a0781 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:29.758941 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45335d9-3977-4144-b31c-a7c0555a0781 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:29.762856 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45335d9-3977-4144-b31c-a7c0555a0781 service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:29.796631 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1050/2103] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:16:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:16:30.622123 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242a8509-42bc-4108-bc75-665810e86ce3 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:30.624981 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-242a8509-42bc-4108-bc75-665810e86ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:16:30.673038 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242a8509-42bc-4108-bc75-665810e86ce3 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:30.691789 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3d8ffd-cfda-4257-a98d-d841f784d71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:30.693037 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3d8ffd-cfda-4257-a98d-d841f784d71e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:30.693613 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3d8ffd-cfda-4257-a98d-d841f784d71e admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:30.694097 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3d8ffd-cfda-4257-a98d-d841f784d71e admin admin] PATH_INFO: `/v3/projects/4f35b48adcff4e0cbc3aa3c7e88acf27` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:30.697896 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3d8ffd-cfda-4257-a98d-d841f784d71e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f35b48adcff4e0cbc3aa3c7e88acf27)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:30.702343 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3d8ffd-cfda-4257-a98d-d841f784d71e admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:30.840372 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:16:30.875281 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-3e3d8ffd-cfda-4257-a98d-d841f784d71e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f35b48adcff4e0cbc3aa3c7e88acf27', 'request_id': 'req-3e3d8ffd-cfda-4257-a98d-d841f784d71e'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:30.903607 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-3e3d8ffd-cfda-4257-a98d-d841f784d71e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f35b48adcff4e0cbc3aa3c7e88acf27', 'request_id': 'req-3e3d8ffd-cfda-4257-a98d-d841f784d71e'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:30.928108 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-3e3d8ffd-cfda-4257-a98d-d841f784d71e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f35b48adcff4e0cbc3aa3c7e88acf27', 'request_id': 'req-3e3d8ffd-cfda-4257-a98d-d841f784d71e'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:30.958929 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-3e3d8ffd-cfda-4257-a98d-d841f784d71e admin admin] The token cache is being invalidate because project 4f35b48adcff4e0cbc3aa3c7e88acf27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81618) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:16:30.959355 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-3e3d8ffd-cfda-4257-a98d-d841f784d71e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:30.961314 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-3e3d8ffd-cfda-4257-a98d-d841f784d71e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:30.963576 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1054/2104] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:16:30 2022] DELETE /identity/v3/projects/4f35b48adcff4e0cbc3aa3c7e88acf27 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:16:30.969513 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45335d9-3977-4144-b31c-a7c0555a0781 service neutron] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:30.985541 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45335d9-3977-4144-b31c-a7c0555a0781 service neutron] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:31.133615 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c41615-8094-4df4-8db0-8ecb162a81b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:31.134264 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c41615-8094-4df4-8db0-8ecb162a81b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:31.134604 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c41615-8094-4df4-8db0-8ecb162a81b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:31.134962 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c41615-8094-4df4-8db0-8ecb162a81b7 admin admin] PATH_INFO: `/v3/users/b29fd62d4e8c484bb2bf640ba44c9280` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:31.136398 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c41615-8094-4df4-8db0-8ecb162a81b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b29fd62d4e8c484bb2bf640ba44c9280)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:31.140094 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c41615-8094-4df4-8db0-8ecb162a81b7 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:31.253139 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-13c41615-8094-4df4-8db0-8ecb162a81b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b29fd62d4e8c484bb2bf640ba44c9280'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:31.264873 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-13c41615-8094-4df4-8db0-8ecb162a81b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b29fd62d4e8c484bb2bf640ba44c9280'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:31.291914 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-13c41615-8094-4df4-8db0-8ecb162a81b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b29fd62d4e8c484bb2bf640ba44c9280'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:31.329263 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-13c41615-8094-4df4-8db0-8ecb162a81b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b29fd62d4e8c484bb2bf640ba44c9280'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:31.331260 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-13c41615-8094-4df4-8db0-8ecb162a81b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b29fd62d4e8c484bb2bf640ba44c9280'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:31.378108 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1051/2105] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:16:30 2022] DELETE /identity/v3/users/b29fd62d4e8c484bb2bf640ba44c9280 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:16:32.114048 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3d8ffd-cfda-4257-a98d-d841f784d71e admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:32.115972 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-3e3d8ffd-cfda-4257-a98d-d841f784d71e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:16:32.131164 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3d8ffd-cfda-4257-a98d-d841f784d71e admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:32.294269 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b091c0-accb-421d-8057-6fb99fff52bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:32.295549 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b091c0-accb-421d-8057-6fb99fff52bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:32.296020 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b091c0-accb-421d-8057-6fb99fff52bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:32.298320 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b091c0-accb-421d-8057-6fb99fff52bf admin admin] PATH_INFO: `/v3/users/11794279b67342e78ff86b6b9d34bf68` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:32.300299 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b091c0-accb-421d-8057-6fb99fff52bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=11794279b67342e78ff86b6b9d34bf68)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:32.302947 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b091c0-accb-421d-8057-6fb99fff52bf admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:32.333415 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c41615-8094-4df4-8db0-8ecb162a81b7 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:32.337427 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-13c41615-8094-4df4-8db0-8ecb162a81b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:16:32.376244 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c41615-8094-4df4-8db0-8ecb162a81b7 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:32.406133 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f56a9e2-dcf8-4d3a-9f7f-c8b2356a5649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:32.407033 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f56a9e2-dcf8-4d3a-9f7f-c8b2356a5649 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:32.407427 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f56a9e2-dcf8-4d3a-9f7f-c8b2356a5649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:32.409306 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f56a9e2-dcf8-4d3a-9f7f-c8b2356a5649 admin admin] PATH_INFO: `/v3/projects/5e9c92e7353d443cb90bc5e22d4ddad9` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:32.419776 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-a7b091c0-accb-421d-8057-6fb99fff52bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11794279b67342e78ff86b6b9d34bf68'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:32.423268 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f56a9e2-dcf8-4d3a-9f7f-c8b2356a5649 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e9c92e7353d443cb90bc5e22d4ddad9)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:32.429362 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f56a9e2-dcf8-4d3a-9f7f-c8b2356a5649 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:32.450332 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-a7b091c0-accb-421d-8057-6fb99fff52bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11794279b67342e78ff86b6b9d34bf68'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:32.484787 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-a7b091c0-accb-421d-8057-6fb99fff52bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11794279b67342e78ff86b6b9d34bf68'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:32.506300 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-a7b091c0-accb-421d-8057-6fb99fff52bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11794279b67342e78ff86b6b9d34bf68'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:32.508592 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-a7b091c0-accb-421d-8057-6fb99fff52bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11794279b67342e78ff86b6b9d34bf68'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:32.524368 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1055/2106] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:16:32 2022] DELETE /identity/v3/users/11794279b67342e78ff86b6b9d34bf68 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:16:32.535182 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe26a53-4998-4ac6-8944-7e1005a73fe6 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:32.535467 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:16:32.535670 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe26a53-4998-4ac6-8944-7e1005a73fe6 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:32.535829 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe26a53-4998-4ac6-8944-7e1005a73fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:32.556370 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-8f56a9e2-dcf8-4d3a-9f7f-c8b2356a5649 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e9c92e7353d443cb90bc5e22d4ddad9', 'request_id': 'req-8f56a9e2-dcf8-4d3a-9f7f-c8b2356a5649'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:32.566415 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-8f56a9e2-dcf8-4d3a-9f7f-c8b2356a5649 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e9c92e7353d443cb90bc5e22d4ddad9', 'request_id': 'req-8f56a9e2-dcf8-4d3a-9f7f-c8b2356a5649'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:32.576773 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-8f56a9e2-dcf8-4d3a-9f7f-c8b2356a5649 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e9c92e7353d443cb90bc5e22d4ddad9', 'request_id': 'req-8f56a9e2-dcf8-4d3a-9f7f-c8b2356a5649'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:32.578606 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-3fe26a53-4998-4ac6-8944-7e1005a73fe6 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:16:32.583923 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-3fe26a53-4998-4ac6-8944-7e1005a73fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:16:32.616440 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-8f56a9e2-dcf8-4d3a-9f7f-c8b2356a5649 admin admin] The token cache is being invalidate because project 5e9c92e7353d443cb90bc5e22d4ddad9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81617) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:16:32.616440 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-8f56a9e2-dcf8-4d3a-9f7f-c8b2356a5649 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:32.621451 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-8f56a9e2-dcf8-4d3a-9f7f-c8b2356a5649 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:32.626631 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1052/2107] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:16:32 2022] DELETE /identity/v3/projects/5e9c92e7353d443cb90bc5e22d4ddad9 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:16:32.632137 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45584bb2-f2fa-4691-9dd6-0f08996eb354 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:32.632137 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45584bb2-f2fa-4691-9dd6-0f08996eb354 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:32.632767 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45584bb2-f2fa-4691-9dd6-0f08996eb354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:32.681320 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-45584bb2-f2fa-4691-9dd6-0f08996eb354 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:16:32.684766 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-45584bb2-f2fa-4691-9dd6-0f08996eb354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:16:32.687206 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:16:32.757765 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1056/2108] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:16:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:16:32.825219 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe26a53-4998-4ac6-8944-7e1005a73fe6 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:32.827514 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-3fe26a53-4998-4ac6-8944-7e1005a73fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:16:32.890435 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1053/2109] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:16:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:16:32.895768 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d732618b-b3a9-40fa-aac8-40bfb68f6396 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:32.896609 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d732618b-b3a9-40fa-aac8-40bfb68f6396 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:32.897237 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d732618b-b3a9-40fa-aac8-40bfb68f6396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:32.966059 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-d732618b-b3a9-40fa-aac8-40bfb68f6396 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:16:32.972645 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-d732618b-b3a9-40fa-aac8-40bfb68f6396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:16:32.994433 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1054/2110] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:16:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:16:32.999523 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d732618b-b3a9-40fa-aac8-40bfb68f6396 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:33.017795 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe26a53-4998-4ac6-8944-7e1005a73fe6 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:33.026508 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411da032-b008-4dc5-9581-d95ecefc4372 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:33.026967 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d732618b-b3a9-40fa-aac8-40bfb68f6396 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:33.027824 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411da032-b008-4dc5-9581-d95ecefc4372 service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:33.029064 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411da032-b008-4dc5-9581-d95ecefc4372 service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:33.029980 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411da032-b008-4dc5-9581-d95ecefc4372 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:33.036745 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782ca20a-c93f-4f20-8f22-4433c93d1aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:33.037798 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782ca20a-c93f-4f20-8f22-4433c93d1aaf admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:33.038312 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782ca20a-c93f-4f20-8f22-4433c93d1aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:33.038849 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782ca20a-c93f-4f20-8f22-4433c93d1aaf admin admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:33.039736 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782ca20a-c93f-4f20-8f22-4433c93d1aaf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:16:33.040273 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782ca20a-c93f-4f20-8f22-4433c93d1aaf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:33.042326 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782ca20a-c93f-4f20-8f22-4433c93d1aaf admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:33.060868 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411da032-b008-4dc5-9581-d95ecefc4372 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:33.063137 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1055/2111] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:16:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:16:33.069883 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782ca20a-c93f-4f20-8f22-4433c93d1aaf admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:33.070286 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411da032-b008-4dc5-9581-d95ecefc4372 service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:33.088019 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782ca20a-c93f-4f20-8f22-4433c93d1aaf admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:33.098674 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1c6596-23ec-4996-8210-e20ce7092110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:33.099873 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c6596-23ec-4996-8210-e20ce7092110 admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:33.100387 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c6596-23ec-4996-8210-e20ce7092110 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:33.100914 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c6596-23ec-4996-8210-e20ce7092110 admin admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:33.101858 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1c6596-23ec-4996-8210-e20ce7092110 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:33.104463 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1c6596-23ec-4996-8210-e20ce7092110 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:33.104873 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1057/2112] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:16:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 16:16:33.156923 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1056/2113] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:16:33 2022] POST /identity/v3/projects => generated 383 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:16:33.163524 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411da032-b008-4dc5-9581-d95ecefc4372 service neutron] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:33.198561 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411da032-b008-4dc5-9581-d95ecefc4372 service neutron] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:33.359583 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b18b6ba-0cb5-426c-9aa0-01991b51913a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:33.360451 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b18b6ba-0cb5-426c-9aa0-01991b51913a admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:33.360809 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b18b6ba-0cb5-426c-9aa0-01991b51913a admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:33.361142 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b18b6ba-0cb5-426c-9aa0-01991b51913a admin admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:33.363062 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b18b6ba-0cb5-426c-9aa0-01991b51913a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:33.365727 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b18b6ba-0cb5-426c-9aa0-01991b51913a admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:33.403669 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1058/2114] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:16:33 2022] POST /identity/v3/users => generated 292 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:16:33.404943 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1c6596-23ec-4996-8210-e20ce7092110 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:33.428486 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1c6596-23ec-4996-8210-e20ce7092110 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:33.437835 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f691a48d-f79b-45d6-81f2-93e350dc096e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:33.438653 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f691a48d-f79b-45d6-81f2-93e350dc096e admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:33.439009 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f691a48d-f79b-45d6-81f2-93e350dc096e admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:33.439414 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f691a48d-f79b-45d6-81f2-93e350dc096e admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:33.441636 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f691a48d-f79b-45d6-81f2-93e350dc096e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:33.443696 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f691a48d-f79b-45d6-81f2-93e350dc096e admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:33.468845 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1057/2115] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:16:33 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:16:33.469461 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b18b6ba-0cb5-426c-9aa0-01991b51913a admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:33.484218 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b18b6ba-0cb5-426c-9aa0-01991b51913a admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:33.490717 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f7387c-8ab2-4c29-a84f-7da6a7eb6f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:33.491730 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f7387c-8ab2-4c29-a84f-7da6a7eb6f7d admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:33.492406 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f7387c-8ab2-4c29-a84f-7da6a7eb6f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:33.492848 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f7387c-8ab2-4c29-a84f-7da6a7eb6f7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:33.493916 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f7387c-8ab2-4c29-a84f-7da6a7eb6f7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:33.508161 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f7387c-8ab2-4c29-a84f-7da6a7eb6f7d admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:33.533547 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1059/2116] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:16:33 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 16:16:33.538223 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f691a48d-f79b-45d6-81f2-93e350dc096e admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:33.563278 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f691a48d-f79b-45d6-81f2-93e350dc096e admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:33.576175 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc29bbb-e89a-4343-9cd9-c77dac9e2d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:33.577620 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc29bbb-e89a-4343-9cd9-c77dac9e2d41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:33.578181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc29bbb-e89a-4343-9cd9-c77dac9e2d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:33.578807 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc29bbb-e89a-4343-9cd9-c77dac9e2d41 admin admin] PATH_INFO: `/v3/projects/42eddecf698341018a42c7fff646c0b5/users/b02c8c5481484deab6938656a31271f5/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:33.616390 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc29bbb-e89a-4343-9cd9-c77dac9e2d41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42eddecf698341018a42c7fff646c0b5, user_id=b02c8c5481484deab6938656a31271f5, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:33.620366 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc29bbb-e89a-4343-9cd9-c77dac9e2d41 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:33.669029 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1058/2117] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:16:33 2022] PUT /identity/v3/projects/42eddecf698341018a42c7fff646c0b5/users/b02c8c5481484deab6938656a31271f5/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:16:33.680099 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818f08c-e17a-48c4-a8cc-a32a64de2bb5 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:33.680775 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818f08c-e17a-48c4-a8cc-a32a64de2bb5 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:33.681463 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818f08c-e17a-48c4-a8cc-a32a64de2bb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:33.750447 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc29bbb-e89a-4343-9cd9-c77dac9e2d41 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:33.752860 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-8fc29bbb-e89a-4343-9cd9-c77dac9e2d41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:16:33.800000 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-5818f08c-e17a-48c4-a8cc-a32a64de2bb5 None None] MFA Rules not processed for user `b02c8c5481484deab6938656a31271f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:16:33.801555 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-5818f08c-e17a-48c4-a8cc-a32a64de2bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:16:33.868928 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:16:33.904464 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc29bbb-e89a-4343-9cd9-c77dac9e2d41 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:33.913465 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cee319-dadf-48ef-b049-182c7f8f5823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:33.918831 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cee319-dadf-48ef-b049-182c7f8f5823 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:33.919149 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cee319-dadf-48ef-b049-182c7f8f5823 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:33.919473 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cee319-dadf-48ef-b049-182c7f8f5823 admin admin] PATH_INFO: `/v3/projects/56d3906965284a8d92ab83e811299d4f` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:33.921849 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cee319-dadf-48ef-b049-182c7f8f5823 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56d3906965284a8d92ab83e811299d4f)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:33.925080 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cee319-dadf-48ef-b049-182c7f8f5823 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:33.948526 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:16:34.043054 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1060/2118] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:16:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:16:34.044664 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5818f08c-e17a-48c4-a8cc-a32a64de2bb5 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:34.070883 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-e8cee319-dadf-48ef-b049-182c7f8f5823 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56d3906965284a8d92ab83e811299d4f', 'request_id': 'req-e8cee319-dadf-48ef-b049-182c7f8f5823'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:34.089568 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-e8cee319-dadf-48ef-b049-182c7f8f5823 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56d3906965284a8d92ab83e811299d4f', 'request_id': 'req-e8cee319-dadf-48ef-b049-182c7f8f5823'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:34.093009 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5818f08c-e17a-48c4-a8cc-a32a64de2bb5 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:34.108483 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-e8cee319-dadf-48ef-b049-182c7f8f5823 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56d3906965284a8d92ab83e811299d4f', 'request_id': 'req-e8cee319-dadf-48ef-b049-182c7f8f5823'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:34.139645 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-e8cee319-dadf-48ef-b049-182c7f8f5823 admin admin] The token cache is being invalidate because project 56d3906965284a8d92ab83e811299d4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81617) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:16:34.139976 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-e8cee319-dadf-48ef-b049-182c7f8f5823 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:34.142587 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-e8cee319-dadf-48ef-b049-182c7f8f5823 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:34.154579 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1059/2119] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:16:33 2022] DELETE /identity/v3/projects/56d3906965284a8d92ab83e811299d4f => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:16:34.156877 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d500f830-0481-4c86-ac98-9dd3e12718be None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:34.157319 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d500f830-0481-4c86-ac98-9dd3e12718be None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:34.157656 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d500f830-0481-4c86-ac98-9dd3e12718be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:34.216393 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-d500f830-0481-4c86-ac98-9dd3e12718be None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:16:34.223610 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-d500f830-0481-4c86-ac98-9dd3e12718be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:16:34.396712 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1060/2120] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:16:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:16:34.399957 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d78498-f48a-4540-9d10-1fe251ff5e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:34.402192 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d78498-f48a-4540-9d10-1fe251ff5e70 admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:34.403034 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d78498-f48a-4540-9d10-1fe251ff5e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:34.403803 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d78498-f48a-4540-9d10-1fe251ff5e70 admin admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:34.405297 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d78498-f48a-4540-9d10-1fe251ff5e70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:34.411975 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e4aec-35df-4089-9fb3-2d5b7efb7dd3 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:34.413578 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e4aec-35df-4089-9fb3-2d5b7efb7dd3 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:34.413935 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e4aec-35df-4089-9fb3-2d5b7efb7dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:34.427221 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d78498-f48a-4540-9d10-1fe251ff5e70 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:34.472766 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-4d6e4aec-35df-4089-9fb3-2d5b7efb7dd3 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:16:34.474878 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-4d6e4aec-35df-4089-9fb3-2d5b7efb7dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:16:34.523089 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1061/2121] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:16:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:16:34.529878 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e4aec-35df-4089-9fb3-2d5b7efb7dd3 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:34.532365 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1061/2122] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:16:34 2022] POST /identity/v3/projects => generated 385 bytes in 484 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:16:34.535603 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d78498-f48a-4540-9d10-1fe251ff5e70 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:34.539374 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-b6d78498-f48a-4540-9d10-1fe251ff5e70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:16:34.545593 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e4aec-35df-4089-9fb3-2d5b7efb7dd3 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:34.639505 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e28544-a4b4-4a19-bb33-e6a0daec1af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:34.641023 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e28544-a4b4-4a19-bb33-e6a0daec1af7 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:34.641023 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e28544-a4b4-4a19-bb33-e6a0daec1af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:34.641023 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e28544-a4b4-4a19-bb33-e6a0daec1af7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:34.641922 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e28544-a4b4-4a19-bb33-e6a0daec1af7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:16:34.642244 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e28544-a4b4-4a19-bb33-e6a0daec1af7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:34.644915 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e28544-a4b4-4a19-bb33-e6a0daec1af7 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:34.666252 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1062/2123] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:16:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:16:34.667188 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e28544-a4b4-4a19-bb33-e6a0daec1af7 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:34.687051 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e28544-a4b4-4a19-bb33-e6a0daec1af7 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:34.704016 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbdacc3-1b40-4ed0-a265-f34ed1c6e121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:34.704636 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbdacc3-1b40-4ed0-a265-f34ed1c6e121 admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:34.704932 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbdacc3-1b40-4ed0-a265-f34ed1c6e121 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:34.705276 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbdacc3-1b40-4ed0-a265-f34ed1c6e121 admin admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:34.705887 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbdacc3-1b40-4ed0-a265-f34ed1c6e121 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:34.711533 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbdacc3-1b40-4ed0-a265-f34ed1c6e121 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:34.727971 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:16:34.779406 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1063/2124] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:16:34 2022] POST /identity/v3/projects => generated 389 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:16:34.786056 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbdacc3-1b40-4ed0-a265-f34ed1c6e121 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:34.869083 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d78498-f48a-4540-9d10-1fe251ff5e70 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:34.897063 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbdacc3-1b40-4ed0-a265-f34ed1c6e121 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:34.939777 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26db3408-b4af-41d0-9183-b955dc673382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:34.940671 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db3408-b4af-41d0-9183-b955dc673382 admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:34.941109 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db3408-b4af-41d0-9183-b955dc673382 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:34.941510 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db3408-b4af-41d0-9183-b955dc673382 admin admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:34.942421 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26db3408-b4af-41d0-9183-b955dc673382 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:34.946128 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26db3408-b4af-41d0-9183-b955dc673382 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:34.992525 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1064/2125] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:16:34 2022] POST /identity/v3/users => generated 295 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:16:34.996998 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26db3408-b4af-41d0-9183-b955dc673382 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:35.030070 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26db3408-b4af-41d0-9183-b955dc673382 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:35.050693 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a8b7b3-9ca0-42ee-b447-81dc6d21efae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:35.052001 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff061c5-2f16-4b71-af96-cd58946a10e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:35.053951 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff061c5-2f16-4b71-af96-cd58946a10e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:35.054388 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a8b7b3-9ca0-42ee-b447-81dc6d21efae admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:35.055110 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a8b7b3-9ca0-42ee-b447-81dc6d21efae admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:35.055595 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff061c5-2f16-4b71-af96-cd58946a10e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:35.056239 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a8b7b3-9ca0-42ee-b447-81dc6d21efae admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:35.057164 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff061c5-2f16-4b71-af96-cd58946a10e6 admin admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:35.057953 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a8b7b3-9ca0-42ee-b447-81dc6d21efae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:35.061091 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff061c5-2f16-4b71-af96-cd58946a10e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:35.063049 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a8b7b3-9ca0-42ee-b447-81dc6d21efae admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:35.066247 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff061c5-2f16-4b71-af96-cd58946a10e6 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:35.091740 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1065/2126] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:16:34 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:16:35.093086 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a8b7b3-9ca0-42ee-b447-81dc6d21efae admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:35.117315 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a8b7b3-9ca0-42ee-b447-81dc6d21efae admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:35.124175 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1062/2127] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:16:34 2022] POST /identity/v3/users => generated 299 bytes in 591 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:16:35.136522 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff061c5-2f16-4b71-af96-cd58946a10e6 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:35.143069 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17fffe1-acdc-43a8-9b6c-a6546fad254c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:35.145051 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17fffe1-acdc-43a8-9b6c-a6546fad254c admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:35.146874 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17fffe1-acdc-43a8-9b6c-a6546fad254c admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:35.147561 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17fffe1-acdc-43a8-9b6c-a6546fad254c admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:35.148825 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17fffe1-acdc-43a8-9b6c-a6546fad254c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:35.152184 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff061c5-2f16-4b71-af96-cd58946a10e6 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:35.157994 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57815461-a4da-4e05-a7cc-26888e3c447a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:35.159315 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17fffe1-acdc-43a8-9b6c-a6546fad254c admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:35.159489 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57815461-a4da-4e05-a7cc-26888e3c447a admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:35.159719 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57815461-a4da-4e05-a7cc-26888e3c447a admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:35.160000 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57815461-a4da-4e05-a7cc-26888e3c447a admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:35.160674 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57815461-a4da-4e05-a7cc-26888e3c447a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:35.162510 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57815461-a4da-4e05-a7cc-26888e3c447a admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:35.178540 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1063/2128] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:16:35 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:16:35.181401 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57815461-a4da-4e05-a7cc-26888e3c447a admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:35.189475 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:16:35.195156 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57815461-a4da-4e05-a7cc-26888e3c447a admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:35.196176 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1066/2129] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:16:35 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:16:35.206559 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17fffe1-acdc-43a8-9b6c-a6546fad254c admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:35.206842 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3433df14-42c8-4353-933d-b0f472240293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:35.211871 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3433df14-42c8-4353-933d-b0f472240293 admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:35.211871 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3433df14-42c8-4353-933d-b0f472240293 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:35.211871 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3433df14-42c8-4353-933d-b0f472240293 admin admin] PATH_INFO: `/v3/projects/cff869172eed44358a1e1ed1f38bce24/users/e57b470916734d2d85e1ffffb47a93b2/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:35.253377 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17fffe1-acdc-43a8-9b6c-a6546fad254c admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:35.254603 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3433df14-42c8-4353-933d-b0f472240293 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cff869172eed44358a1e1ed1f38bce24, user_id=e57b470916734d2d85e1ffffb47a93b2, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:35.257872 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3433df14-42c8-4353-933d-b0f472240293 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:35.261978 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e87f049-202e-43b3-a5a2-03a4fc962bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:35.263319 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e87f049-202e-43b3-a5a2-03a4fc962bc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:35.263897 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e87f049-202e-43b3-a5a2-03a4fc962bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:35.264645 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e87f049-202e-43b3-a5a2-03a4fc962bc1 admin admin] PATH_INFO: `/v3/projects/af2f5f52ab5a45d7a38ba24071905110/users/a1387899781d43e9a5e6201fd137715d/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:35.286596 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1064/2130] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:16:35 2022] PUT /identity/v3/projects/cff869172eed44358a1e1ed1f38bce24/users/e57b470916734d2d85e1ffffb47a93b2/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:16:35.292520 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3433df14-42c8-4353-933d-b0f472240293 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:35.300893 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e87f049-202e-43b3-a5a2-03a4fc962bc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af2f5f52ab5a45d7a38ba24071905110, user_id=a1387899781d43e9a5e6201fd137715d, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:35.307154 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e87f049-202e-43b3-a5a2-03a4fc962bc1 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:35.333470 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1067/2131] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:16:35 2022] PUT /identity/v3/projects/af2f5f52ab5a45d7a38ba24071905110/users/a1387899781d43e9a5e6201fd137715d/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:16:35.356950 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd715bf0-63ae-41ca-be37-f6ca1b5c1387 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:35.357568 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd715bf0-63ae-41ca-be37-f6ca1b5c1387 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:35.358016 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd715bf0-63ae-41ca-be37-f6ca1b5c1387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:35.436002 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-bd715bf0-63ae-41ca-be37-f6ca1b5c1387 None None] MFA Rules not processed for user `a1387899781d43e9a5e6201fd137715d`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:16:35.438149 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3433df14-42c8-4353-933d-b0f472240293 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:35.438840 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-bd715bf0-63ae-41ca-be37-f6ca1b5c1387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:16:35.576626 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1068/2132] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:16:35 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:16:35.587150 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbf9ac5-bb86-4bdf-a946-6104a31a9ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:35.587872 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbf9ac5-bb86-4bdf-a946-6104a31a9ec9 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:35.588223 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbf9ac5-bb86-4bdf-a946-6104a31a9ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:35.588567 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbf9ac5-bb86-4bdf-a946-6104a31a9ec9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:35.589305 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbf9ac5-bb86-4bdf-a946-6104a31a9ec9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:35.590987 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd715bf0-63ae-41ca-be37-f6ca1b5c1387 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:35.591302 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbf9ac5-bb86-4bdf-a946-6104a31a9ec9 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:35.593155 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-bd715bf0-63ae-41ca-be37-f6ca1b5c1387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:16:35.609951 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1065/2133] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:16:35 2022] GET /identity/v3/roles => generated 1420 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:16:35.612023 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbf9ac5-bb86-4bdf-a946-6104a31a9ec9 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:35.621320 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:16:35.647366 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbf9ac5-bb86-4bdf-a946-6104a31a9ec9 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:35.660118 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab09605-382c-4d6e-977c-907fd450a444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:35.663780 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab09605-382c-4d6e-977c-907fd450a444 admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:35.666334 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab09605-382c-4d6e-977c-907fd450a444 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:35.666851 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab09605-382c-4d6e-977c-907fd450a444 admin admin] PATH_INFO: `/v3/projects/cff869172eed44358a1e1ed1f38bce24/users/e57b470916734d2d85e1ffffb47a93b2/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:35.673553 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd715bf0-63ae-41ca-be37-f6ca1b5c1387 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:35.678732 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8068e877-16e3-4b0f-87c2-385d9b807ecb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:35.679478 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8068e877-16e3-4b0f-87c2-385d9b807ecb service cinder] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:35.679828 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8068e877-16e3-4b0f-87c2-385d9b807ecb service cinder] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:35.680299 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab09605-382c-4d6e-977c-907fd450a444 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cff869172eed44358a1e1ed1f38bce24, user_id=e57b470916734d2d85e1ffffb47a93b2, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:35.680488 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8068e877-16e3-4b0f-87c2-385d9b807ecb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:35.688431 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab09605-382c-4d6e-977c-907fd450a444 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:35.691519 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8068e877-16e3-4b0f-87c2-385d9b807ecb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:35.693261 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8068e877-16e3-4b0f-87c2-385d9b807ecb service cinder] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:35.721477 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1069/2135] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:16:35 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 16:16:35.737410 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1066/2135] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:16:35 2022] PUT /identity/v3/projects/cff869172eed44358a1e1ed1f38bce24/users/e57b470916734d2d85e1ffffb47a93b2/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:16:35.738155 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad2e564-063b-4bb3-b822-b4794274b7bf None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:35.738562 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad2e564-063b-4bb3-b822-b4794274b7bf None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:35.738909 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad2e564-063b-4bb3-b822-b4794274b7bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:35.795783 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-6ad2e564-063b-4bb3-b822-b4794274b7bf None None] MFA Rules not processed for user `e57b470916734d2d85e1ffffb47a93b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:16:35.797842 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-6ad2e564-063b-4bb3-b822-b4794274b7bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:16:35.932589 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1070/2136] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:16:35 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:16:35.941889 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab09605-382c-4d6e-977c-907fd450a444 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:35.943682 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-9ab09605-382c-4d6e-977c-907fd450a444 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:16:36.096296 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab09605-382c-4d6e-977c-907fd450a444 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:36.110828 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2881c468-b8f8-4ec9-9788-b6aacd590996 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:36.112879 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2881c468-b8f8-4ec9-9788-b6aacd590996 service cinder] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:36.113216 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2881c468-b8f8-4ec9-9788-b6aacd590996 service cinder] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:36.113762 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2881c468-b8f8-4ec9-9788-b6aacd590996 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:36.120218 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-2881c468-b8f8-4ec9-9788-b6aacd590996 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:16:36.183961 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:16:36.317211 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2881c468-b8f8-4ec9-9788-b6aacd590996 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:36.319069 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2881c468-b8f8-4ec9-9788-b6aacd590996 service cinder] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:36.354515 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1067/2137] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:16:35 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:16:38.835737 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586b3504-ffaf-4182-a720-c2f6e776c8fb None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:38.836488 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586b3504-ffaf-4182-a720-c2f6e776c8fb None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:38.837887 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586b3504-ffaf-4182-a720-c2f6e776c8fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:38.878289 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-586b3504-ffaf-4182-a720-c2f6e776c8fb None None] MFA Rules not processed for user `7222bd3b313b47e5a4fa0ebc9ff0cdd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:16:38.880848 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-586b3504-ffaf-4182-a720-c2f6e776c8fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:16:39.014439 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1071/2138] 10.222.0.57 () {58 vars in 967 bytes} [Mon Sep 19 16:16:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:16:39.041322 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2881c468-b8f8-4ec9-9788-b6aacd590996 service cinder] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:39.043359 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-2881c468-b8f8-4ec9-9788-b6aacd590996 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:16:39.183047 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2881c468-b8f8-4ec9-9788-b6aacd590996 service cinder] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:39.192257 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914c76bd-1e52-4c24-a3d9-4e68fad6764d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '884a0524aca744b99c7496c11c3291ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:39.193188 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914c76bd-1e52-4c24-a3d9-4e68fad6764d service swift] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:39.193661 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914c76bd-1e52-4c24-a3d9-4e68fad6764d service swift] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:39.194176 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914c76bd-1e52-4c24-a3d9-4e68fad6764d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:39.211275 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914c76bd-1e52-4c24-a3d9-4e68fad6764d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:39.214341 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914c76bd-1e52-4c24-a3d9-4e68fad6764d service swift] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:39.229250 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1068/2139] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Sep 19 16:16:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 16:16:42.728372 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586b3504-ffaf-4182-a720-c2f6e776c8fb None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:42.735311 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-586b3504-ffaf-4182-a720-c2f6e776c8fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:16:42.910759 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:16:42.959439 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586b3504-ffaf-4182-a720-c2f6e776c8fb None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:42.967720 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eeb872-6fb1-440b-a862-8c00a8f137a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:42.969873 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eeb872-6fb1-440b-a862-8c00a8f137a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:42.969873 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eeb872-6fb1-440b-a862-8c00a8f137a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:42.969873 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eeb872-6fb1-440b-a862-8c00a8f137a8 admin admin] PATH_INFO: `/v3/users/fe870f5b19334b47b972d7655876472e` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:42.972544 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eeb872-6fb1-440b-a862-8c00a8f137a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe870f5b19334b47b972d7655876472e)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:42.976714 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eeb872-6fb1-440b-a862-8c00a8f137a8 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:43.066074 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-c8eeb872-6fb1-440b-a862-8c00a8f137a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe870f5b19334b47b972d7655876472e'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:43.071239 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-c8eeb872-6fb1-440b-a862-8c00a8f137a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe870f5b19334b47b972d7655876472e'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:43.086658 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-c8eeb872-6fb1-440b-a862-8c00a8f137a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe870f5b19334b47b972d7655876472e'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:43.105924 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-c8eeb872-6fb1-440b-a862-8c00a8f137a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe870f5b19334b47b972d7655876472e'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:43.107239 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-c8eeb872-6fb1-440b-a862-8c00a8f137a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe870f5b19334b47b972d7655876472e'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:43.138487 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1072/2140] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:16:42 2022] DELETE /identity/v3/users/fe870f5b19334b47b972d7655876472e => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:16:43.638608 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914c76bd-1e52-4c24-a3d9-4e68fad6764d service swift] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:43.651790 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914c76bd-1e52-4c24-a3d9-4e68fad6764d service swift] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:43.768267 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913f8b65-5d4e-4da9-8cb8-b2e2f80bdb5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:43.769128 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f8b65-5d4e-4da9-8cb8-b2e2f80bdb5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:43.769501 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f8b65-5d4e-4da9-8cb8-b2e2f80bdb5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:43.769951 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f8b65-5d4e-4da9-8cb8-b2e2f80bdb5f admin admin] PATH_INFO: `/v3/projects/0e641d48680849328816261ffd5b3b42` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:43.771486 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913f8b65-5d4e-4da9-8cb8-b2e2f80bdb5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e641d48680849328816261ffd5b3b42)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:43.773814 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913f8b65-5d4e-4da9-8cb8-b2e2f80bdb5f admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:43.832452 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-913f8b65-5d4e-4da9-8cb8-b2e2f80bdb5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e641d48680849328816261ffd5b3b42', 'request_id': 'req-913f8b65-5d4e-4da9-8cb8-b2e2f80bdb5f'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:43.840084 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-913f8b65-5d4e-4da9-8cb8-b2e2f80bdb5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e641d48680849328816261ffd5b3b42', 'request_id': 'req-913f8b65-5d4e-4da9-8cb8-b2e2f80bdb5f'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:43.841622 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:16:43.850545 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-913f8b65-5d4e-4da9-8cb8-b2e2f80bdb5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e641d48680849328816261ffd5b3b42', 'request_id': 'req-913f8b65-5d4e-4da9-8cb8-b2e2f80bdb5f'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:43.868910 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-913f8b65-5d4e-4da9-8cb8-b2e2f80bdb5f admin admin] The token cache is being invalidate because project 0e641d48680849328816261ffd5b3b42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81618) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:16:43.869281 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-913f8b65-5d4e-4da9-8cb8-b2e2f80bdb5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:43.870020 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-913f8b65-5d4e-4da9-8cb8-b2e2f80bdb5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:43.871680 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1069/2141] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:16:43 2022] DELETE /identity/v3/projects/0e641d48680849328816261ffd5b3b42 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 16:16:44.919240 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eeb872-6fb1-440b-a862-8c00a8f137a8 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:44.921156 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-c8eeb872-6fb1-440b-a862-8c00a8f137a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:16:44.990343 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eeb872-6fb1-440b-a862-8c00a8f137a8 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:45.054175 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ec57bd-bc16-47e7-be11-dc302012a1d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:45.054933 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec57bd-bc16-47e7-be11-dc302012a1d1 service cinder] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:45.055306 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec57bd-bc16-47e7-be11-dc302012a1d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:45.055693 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec57bd-bc16-47e7-be11-dc302012a1d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:45.057687 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-95ec57bd-bc16-47e7-be11-dc302012a1d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:16:45.117876 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:16:45.144970 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ec57bd-bc16-47e7-be11-dc302012a1d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:45.146495 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ec57bd-bc16-47e7-be11-dc302012a1d1 service cinder] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:45.158542 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1073/2142] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:16:44 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:16:50.867337 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9fa1ca-b342-4df8-a1b7-23d0c3183082 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:50.868194 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9fa1ca-b342-4df8-a1b7-23d0c3183082 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:50.868194 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9fa1ca-b342-4df8-a1b7-23d0c3183082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:50.914594 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-8e9fa1ca-b342-4df8-a1b7-23d0c3183082 None None] MFA Rules not processed for user `7222bd3b313b47e5a4fa0ebc9ff0cdd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:16:50.916502 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-8e9fa1ca-b342-4df8-a1b7-23d0c3183082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:16:51.054679 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1070/2143] 10.222.0.57 () {58 vars in 1009 bytes} [Mon Sep 19 16:16:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:16:51.071650 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ec57bd-bc16-47e7-be11-dc302012a1d1 service cinder] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:51.073234 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-95ec57bd-bc16-47e7-be11-dc302012a1d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:16:51.191513 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ec57bd-bc16-47e7-be11-dc302012a1d1 service cinder] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:51.199368 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da4bf9d-b895-4e3e-9589-3fde3d2e04e1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '884a0524aca744b99c7496c11c3291ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:51.200103 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da4bf9d-b895-4e3e-9589-3fde3d2e04e1 service swift] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:51.200411 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da4bf9d-b895-4e3e-9589-3fde3d2e04e1 service swift] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:51.200727 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da4bf9d-b895-4e3e-9589-3fde3d2e04e1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:51.211640 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da4bf9d-b895-4e3e-9589-3fde3d2e04e1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:51.213779 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da4bf9d-b895-4e3e-9589-3fde3d2e04e1 service swift] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:51.225341 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1074/2144] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Sep 19 16:16:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 19 16:16:54.186303 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9fa1ca-b342-4df8-a1b7-23d0c3183082 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:54.190319 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-8e9fa1ca-b342-4df8-a1b7-23d0c3183082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:16:54.429630 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da4bf9d-b895-4e3e-9589-3fde3d2e04e1 service swift] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:54.432616 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-5da4bf9d-b895-4e3e-9589-3fde3d2e04e1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:16:54.532251 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9fa1ca-b342-4df8-a1b7-23d0c3183082 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:54.559278 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54271a21-b5a5-49ff-9337-79dcbc5b8f26 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1a985495ed48a4acc48e2e6fc7f977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:54.563644 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54271a21-b5a5-49ff-9337-79dcbc5b8f26 service placement] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:54.564212 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54271a21-b5a5-49ff-9337-79dcbc5b8f26 service placement] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:54.564724 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54271a21-b5a5-49ff-9337-79dcbc5b8f26 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:54.575791 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-54271a21-b5a5-49ff-9337-79dcbc5b8f26 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:16:54.641438 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54271a21-b5a5-49ff-9337-79dcbc5b8f26 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:54.644646 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54271a21-b5a5-49ff-9337-79dcbc5b8f26 service placement] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:54.676562 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da4bf9d-b895-4e3e-9589-3fde3d2e04e1 service swift] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:54.690279 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1071/2145] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:16:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:16:54.692910 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54271a21-b5a5-49ff-9337-79dcbc5b8f26 service placement] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:54.695788 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-54271a21-b5a5-49ff-9337-79dcbc5b8f26 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:16:54.715454 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f3b6c0-6bf5-4a73-bdb6-653461b01f87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:54.724902 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f3b6c0-6bf5-4a73-bdb6-653461b01f87 service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:54.727321 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f3b6c0-6bf5-4a73-bdb6-653461b01f87 service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:54.728157 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f3b6c0-6bf5-4a73-bdb6-653461b01f87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:54.744127 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-96f3b6c0-6bf5-4a73-bdb6-653461b01f87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:16:54.941579 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54271a21-b5a5-49ff-9337-79dcbc5b8f26 service placement] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:54.964368 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84c779d-959b-4653-9e07-5adabb13e20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:54.966993 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c779d-959b-4653-9e07-5adabb13e20f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:54.967473 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c779d-959b-4653-9e07-5adabb13e20f admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:54.967958 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c779d-959b-4653-9e07-5adabb13e20f admin admin] PATH_INFO: `/v3/users/b02c8c5481484deab6938656a31271f5` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:54.977849 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84c779d-959b-4653-9e07-5adabb13e20f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b02c8c5481484deab6938656a31271f5)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:54.986768 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84c779d-959b-4653-9e07-5adabb13e20f admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:55.101734 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:16:55.130702 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f3b6c0-6bf5-4a73-bdb6-653461b01f87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:55.142855 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f3b6c0-6bf5-4a73-bdb6-653461b01f87 service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:55.206496 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1075/2146] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:16:54 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 778 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 16:16:55.226346 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-a84c779d-959b-4653-9e07-5adabb13e20f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b02c8c5481484deab6938656a31271f5'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:55.316669 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-a84c779d-959b-4653-9e07-5adabb13e20f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b02c8c5481484deab6938656a31271f5'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:55.360039 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-a84c779d-959b-4653-9e07-5adabb13e20f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b02c8c5481484deab6938656a31271f5'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:55.379762 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-a84c779d-959b-4653-9e07-5adabb13e20f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b02c8c5481484deab6938656a31271f5'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:55.383503 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-a84c779d-959b-4653-9e07-5adabb13e20f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b02c8c5481484deab6938656a31271f5'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:55.398747 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1072/2147] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:16:54 2022] DELETE /identity/v3/users/b02c8c5481484deab6938656a31271f5 => generated 0 bytes in 707 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:16:55.404896 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab812bb2-695c-4362-88bb-0f026247ff33 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:55.405473 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab812bb2-695c-4362-88bb-0f026247ff33 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:55.405905 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab812bb2-695c-4362-88bb-0f026247ff33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:55.467098 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-ab812bb2-695c-4362-88bb-0f026247ff33 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:16:55.470451 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-ab812bb2-695c-4362-88bb-0f026247ff33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:16:55.491607 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:16:55.655398 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1076/2148] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:16:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:16:55.666784 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84c779d-959b-4653-9e07-5adabb13e20f admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:55.668909 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-a84c779d-959b-4653-9e07-5adabb13e20f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:16:55.815300 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84c779d-959b-4653-9e07-5adabb13e20f admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:55.824836 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade1e0f7-7c1a-4af7-90c9-49759a70920a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:55.826677 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade1e0f7-7c1a-4af7-90c9-49759a70920a service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:55.828692 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade1e0f7-7c1a-4af7-90c9-49759a70920a service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:55.829767 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade1e0f7-7c1a-4af7-90c9-49759a70920a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:55.857083 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade1e0f7-7c1a-4af7-90c9-49759a70920a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:55.861712 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade1e0f7-7c1a-4af7-90c9-49759a70920a service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:55.913519 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1073/2149] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:16:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 16:16:56.696751 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab812bb2-695c-4362-88bb-0f026247ff33 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:56.700805 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-ab812bb2-695c-4362-88bb-0f026247ff33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:16:56.749811 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab812bb2-695c-4362-88bb-0f026247ff33 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:56.765807 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608de826-8907-447f-bbc3-04a2a2a7b509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:56.766800 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608de826-8907-447f-bbc3-04a2a2a7b509 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:56.767271 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608de826-8907-447f-bbc3-04a2a2a7b509 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:56.768623 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608de826-8907-447f-bbc3-04a2a2a7b509 admin admin] PATH_INFO: `/v3/users/a1387899781d43e9a5e6201fd137715d` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:56.770908 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608de826-8907-447f-bbc3-04a2a2a7b509 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1387899781d43e9a5e6201fd137715d)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:56.774946 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608de826-8907-447f-bbc3-04a2a2a7b509 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:56.930018 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade1e0f7-7c1a-4af7-90c9-49759a70920a service neutron] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:56.932464 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-ade1e0f7-7c1a-4af7-90c9-49759a70920a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:16:56.975877 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade1e0f7-7c1a-4af7-90c9-49759a70920a service neutron] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:57.006522 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea038ab1-4d72-4a94-879c-853c18734af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:57.010195 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea038ab1-4d72-4a94-879c-853c18734af1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:57.012801 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea038ab1-4d72-4a94-879c-853c18734af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:57.013475 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea038ab1-4d72-4a94-879c-853c18734af1 admin admin] PATH_INFO: `/v3/projects/42eddecf698341018a42c7fff646c0b5` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:57.019072 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea038ab1-4d72-4a94-879c-853c18734af1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42eddecf698341018a42c7fff646c0b5)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:57.024796 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea038ab1-4d72-4a94-879c-853c18734af1 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:57.076217 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-608de826-8907-447f-bbc3-04a2a2a7b509 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1387899781d43e9a5e6201fd137715d'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:57.081194 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-608de826-8907-447f-bbc3-04a2a2a7b509 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1387899781d43e9a5e6201fd137715d'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:57.116654 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-608de826-8907-447f-bbc3-04a2a2a7b509 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1387899781d43e9a5e6201fd137715d'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:57.211756 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-608de826-8907-447f-bbc3-04a2a2a7b509 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1387899781d43e9a5e6201fd137715d'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:57.213552 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-608de826-8907-447f-bbc3-04a2a2a7b509 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1387899781d43e9a5e6201fd137715d'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:57.252768 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1077/2150] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:16:56 2022] DELETE /identity/v3/users/a1387899781d43e9a5e6201fd137715d => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:16:57.260708 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885a89c7-2d48-46b0-ac60-85f5b54c5055 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:57.261042 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885a89c7-2d48-46b0-ac60-85f5b54c5055 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:57.261382 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885a89c7-2d48-46b0-ac60-85f5b54c5055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:57.318129 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-ea038ab1-4d72-4a94-879c-853c18734af1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42eddecf698341018a42c7fff646c0b5', 'request_id': 'req-ea038ab1-4d72-4a94-879c-853c18734af1'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:57.328431 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-885a89c7-2d48-46b0-ac60-85f5b54c5055 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:16:57.339143 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-885a89c7-2d48-46b0-ac60-85f5b54c5055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:16:57.356836 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-ea038ab1-4d72-4a94-879c-853c18734af1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42eddecf698341018a42c7fff646c0b5', 'request_id': 'req-ea038ab1-4d72-4a94-879c-853c18734af1'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:57.389989 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-ea038ab1-4d72-4a94-879c-853c18734af1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42eddecf698341018a42c7fff646c0b5', 'request_id': 'req-ea038ab1-4d72-4a94-879c-853c18734af1'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:57.427044 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-ea038ab1-4d72-4a94-879c-853c18734af1 admin admin] The token cache is being invalidate because project 42eddecf698341018a42c7fff646c0b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81618) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:16:57.429008 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-ea038ab1-4d72-4a94-879c-853c18734af1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:57.435738 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-ea038ab1-4d72-4a94-879c-853c18734af1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:57.457411 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1074/2151] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:16:56 2022] DELETE /identity/v3/projects/42eddecf698341018a42c7fff646c0b5 => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 16:16:57.461706 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea038ab1-4d72-4a94-879c-853c18734af1 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:57.468167 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-ea038ab1-4d72-4a94-879c-853c18734af1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:16:57.526505 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1078/2152] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:16:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:16:57.570113 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885a89c7-2d48-46b0-ac60-85f5b54c5055 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:57.574222 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-885a89c7-2d48-46b0-ac60-85f5b54c5055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:16:57.603574 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:16:57.708663 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:16:57.785919 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885a89c7-2d48-46b0-ac60-85f5b54c5055 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:57.790245 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157c03db-a452-4579-adbc-2dbe1e2467fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:57.791040 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea038ab1-4d72-4a94-879c-853c18734af1 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:57.791228 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c03db-a452-4579-adbc-2dbe1e2467fb service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:57.791746 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c03db-a452-4579-adbc-2dbe1e2467fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:57.792098 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c03db-a452-4579-adbc-2dbe1e2467fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:57.819697 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9171b64d-1d04-497c-87e6-1db67cffb766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:57.821235 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171b64d-1d04-497c-87e6-1db67cffb766 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:57.821985 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171b64d-1d04-497c-87e6-1db67cffb766 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:57.822780 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171b64d-1d04-497c-87e6-1db67cffb766 admin admin] PATH_INFO: `/v3/users/e57b470916734d2d85e1ffffb47a93b2` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:57.826364 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157c03db-a452-4579-adbc-2dbe1e2467fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:57.828093 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157c03db-a452-4579-adbc-2dbe1e2467fb service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:57.828270 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9171b64d-1d04-497c-87e6-1db67cffb766 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e57b470916734d2d85e1ffffb47a93b2)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:57.839516 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9171b64d-1d04-497c-87e6-1db67cffb766 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:57.854903 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1079/2153] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:16:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 16:16:58.010064 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-9171b64d-1d04-497c-87e6-1db67cffb766 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e57b470916734d2d85e1ffffb47a93b2'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:58.041678 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-9171b64d-1d04-497c-87e6-1db67cffb766 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e57b470916734d2d85e1ffffb47a93b2'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:58.099583 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-9171b64d-1d04-497c-87e6-1db67cffb766 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e57b470916734d2d85e1ffffb47a93b2'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:58.124803 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-9171b64d-1d04-497c-87e6-1db67cffb766 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e57b470916734d2d85e1ffffb47a93b2'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:58.126016 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-9171b64d-1d04-497c-87e6-1db67cffb766 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e57b470916734d2d85e1ffffb47a93b2'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:58.148982 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1075/2154] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:16:57 2022] DELETE /identity/v3/users/e57b470916734d2d85e1ffffb47a93b2 => generated 0 bytes in 684 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 16:16:58.758753 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157c03db-a452-4579-adbc-2dbe1e2467fb service neutron] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:58.786589 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157c03db-a452-4579-adbc-2dbe1e2467fb service neutron] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:58.969529 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8fa09c-13db-4e99-9596-d99984e8ff27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:58.970777 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8fa09c-13db-4e99-9596-d99984e8ff27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:58.973494 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8fa09c-13db-4e99-9596-d99984e8ff27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:58.974046 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8fa09c-13db-4e99-9596-d99984e8ff27 admin admin] PATH_INFO: `/v3/projects/cff869172eed44358a1e1ed1f38bce24` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:58.980198 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8fa09c-13db-4e99-9596-d99984e8ff27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cff869172eed44358a1e1ed1f38bce24)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:58.987652 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8fa09c-13db-4e99-9596-d99984e8ff27 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:59.104559 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-ac8fa09c-13db-4e99-9596-d99984e8ff27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cff869172eed44358a1e1ed1f38bce24', 'request_id': 'req-ac8fa09c-13db-4e99-9596-d99984e8ff27'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:59.113921 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-ac8fa09c-13db-4e99-9596-d99984e8ff27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cff869172eed44358a1e1ed1f38bce24', 'request_id': 'req-ac8fa09c-13db-4e99-9596-d99984e8ff27'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:59.123256 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-ac8fa09c-13db-4e99-9596-d99984e8ff27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cff869172eed44358a1e1ed1f38bce24', 'request_id': 'req-ac8fa09c-13db-4e99-9596-d99984e8ff27'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:59.175133 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-ac8fa09c-13db-4e99-9596-d99984e8ff27 admin admin] The token cache is being invalidate because project cff869172eed44358a1e1ed1f38bce24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81617) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:16:59.175789 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-ac8fa09c-13db-4e99-9596-d99984e8ff27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:59.182115 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-ac8fa09c-13db-4e99-9596-d99984e8ff27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:16:59.184787 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1080/2155] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:16:58 2022] DELETE /identity/v3/projects/cff869172eed44358a1e1ed1f38bce24 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:16:59.193226 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28efc664-b89c-4d71-93bb-94a9890668b7 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:59.193765 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28efc664-b89c-4d71-93bb-94a9890668b7 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:59.194811 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28efc664-b89c-4d71-93bb-94a9890668b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:59.268402 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-28efc664-b89c-4d71-93bb-94a9890668b7 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:16:59.273999 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-28efc664-b89c-4d71-93bb-94a9890668b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:16:59.446572 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1076/2156] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:16:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:16:59.459267 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611e56dd-1a39-4b91-902f-f739cec38152 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:59.460703 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611e56dd-1a39-4b91-902f-f739cec38152 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:59.461357 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611e56dd-1a39-4b91-902f-f739cec38152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:59.512672 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-611e56dd-1a39-4b91-902f-f739cec38152 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:16:59.515147 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-611e56dd-1a39-4b91-902f-f739cec38152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:16:59.537091 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1081/2157] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:16:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:16:59.541718 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28efc664-b89c-4d71-93bb-94a9890668b7 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:59.569104 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28efc664-b89c-4d71-93bb-94a9890668b7 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:59.582854 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a6043-88b2-4d14-a62e-2ffc6e4bd582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:59.584212 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a6043-88b2-4d14-a62e-2ffc6e4bd582 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:59.585147 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a6043-88b2-4d14-a62e-2ffc6e4bd582 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:59.585463 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a6043-88b2-4d14-a62e-2ffc6e4bd582 admin admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:59.591158 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5a6043-88b2-4d14-a62e-2ffc6e4bd582 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:16:59.592185 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5a6043-88b2-4d14-a62e-2ffc6e4bd582 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:59.596607 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5a6043-88b2-4d14-a62e-2ffc6e4bd582 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:59.625756 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1077/2158] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:16:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:16:59.631837 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611e56dd-1a39-4b91-902f-f739cec38152 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:59.661692 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611e56dd-1a39-4b91-902f-f739cec38152 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:59.674728 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14a0622-2121-413b-9954-1e5a428a4a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:59.675815 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a0622-2121-413b-9954-1e5a428a4a2c admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:59.678597 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a0622-2121-413b-9954-1e5a428a4a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:59.679108 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a0622-2121-413b-9954-1e5a428a4a2c admin admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:59.680569 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14a0622-2121-413b-9954-1e5a428a4a2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:59.684909 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14a0622-2121-413b-9954-1e5a428a4a2c admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:59.758539 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1082/2159] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:16:59 2022] POST /identity/v3/projects => generated 373 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 16:16:59.759899 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a6043-88b2-4d14-a62e-2ffc6e4bd582 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:59.774742 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a6043-88b2-4d14-a62e-2ffc6e4bd582 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:16:59.827902 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:16:59.880598 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3887987f-046b-4790-bea7-4772d6a54625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:16:59.881354 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887987f-046b-4790-bea7-4772d6a54625 admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:16:59.881788 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887987f-046b-4790-bea7-4772d6a54625 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:16:59.882164 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887987f-046b-4790-bea7-4772d6a54625 admin admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:16:59.882944 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3887987f-046b-4790-bea7-4772d6a54625 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:16:59.885054 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3887987f-046b-4790-bea7-4772d6a54625 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:16:59.955708 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1078/2160] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:16:59 2022] POST /identity/v3/users => generated 287 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:16:59.956777 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14a0622-2121-413b-9954-1e5a428a4a2c admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:16:59.994656 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14a0622-2121-413b-9954-1e5a428a4a2c admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:17:00.004248 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b3567e-9b22-4844-94eb-d20f93c812ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:17:00.005329 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b3567e-9b22-4844-94eb-d20f93c812ec admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:17:00.005845 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b3567e-9b22-4844-94eb-d20f93c812ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:17:00.006346 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b3567e-9b22-4844-94eb-d20f93c812ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:17:00.007636 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b3567e-9b22-4844-94eb-d20f93c812ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:17:00.010863 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b3567e-9b22-4844-94eb-d20f93c812ec admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:17:00.032403 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1083/2161] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:16:59 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:17:00.038804 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3887987f-046b-4790-bea7-4772d6a54625 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:17:00.054425 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3887987f-046b-4790-bea7-4772d6a54625 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:17:00.066914 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0118e3d5-f8c3-4158-9c92-d66087a0ce52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:17:00.067685 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0118e3d5-f8c3-4158-9c92-d66087a0ce52 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:17:00.068021 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0118e3d5-f8c3-4158-9c92-d66087a0ce52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:17:00.068429 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0118e3d5-f8c3-4158-9c92-d66087a0ce52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:17:00.070203 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0118e3d5-f8c3-4158-9c92-d66087a0ce52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:17:00.081073 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0118e3d5-f8c3-4158-9c92-d66087a0ce52 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:17:00.107849 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1079/2162] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:17:00 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 16:17:00.113744 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b3567e-9b22-4844-94eb-d20f93c812ec admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:17:00.137711 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b3567e-9b22-4844-94eb-d20f93c812ec admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:17:00.146934 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4e35af-9176-4a75-9f9a-3f1ebde5e93d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:17:00.147883 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4e35af-9176-4a75-9f9a-3f1ebde5e93d admin admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:17:00.148270 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4e35af-9176-4a75-9f9a-3f1ebde5e93d admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:17:00.148676 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4e35af-9176-4a75-9f9a-3f1ebde5e93d admin admin] PATH_INFO: `/v3/projects/5fcd58a1bf714f9fb9a5a802ae721596/users/c264d818ff5c466685cc85aa58a77f32/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:17:00.195326 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4e35af-9176-4a75-9f9a-3f1ebde5e93d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fcd58a1bf714f9fb9a5a802ae721596, user_id=c264d818ff5c466685cc85aa58a77f32, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:17:00.204398 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4e35af-9176-4a75-9f9a-3f1ebde5e93d admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:17:00.264399 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:17:00.301493 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1084/2163] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:17:00 2022] PUT /identity/v3/projects/5fcd58a1bf714f9fb9a5a802ae721596/users/c264d818ff5c466685cc85aa58a77f32/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:17:00.308558 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dbadbc-9b1b-4522-8853-0e656d8c1143 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:17:00.309165 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dbadbc-9b1b-4522-8853-0e656d8c1143 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:17:00.309667 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dbadbc-9b1b-4522-8853-0e656d8c1143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:17:00.389587 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4e35af-9176-4a75-9f9a-3f1ebde5e93d admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:17:00.392665 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-cd4e35af-9176-4a75-9f9a-3f1ebde5e93d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:17:00.462567 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-10dbadbc-9b1b-4522-8853-0e656d8c1143 None None] MFA Rules not processed for user `c264d818ff5c466685cc85aa58a77f32`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:17:00.465011 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-10dbadbc-9b1b-4522-8853-0e656d8c1143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:17:00.582235 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4e35af-9176-4a75-9f9a-3f1ebde5e93d admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:17:00.597142 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5f5d66-5291-4f34-9a84-02c6757395ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:17:00.597810 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5f5d66-5291-4f34-9a84-02c6757395ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:17:00.598093 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5f5d66-5291-4f34-9a84-02c6757395ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:17:00.598408 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5f5d66-5291-4f34-9a84-02c6757395ed admin admin] PATH_INFO: `/v3/projects/af2f5f52ab5a45d7a38ba24071905110` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:17:00.599724 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5f5d66-5291-4f34-9a84-02c6757395ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=af2f5f52ab5a45d7a38ba24071905110)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:17:00.601691 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5f5d66-5291-4f34-9a84-02c6757395ed admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:17:00.692263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1080/2164] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:17:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 381 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:17:00.712273 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dbadbc-9b1b-4522-8853-0e656d8c1143 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:17:00.717372 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-10dbadbc-9b1b-4522-8853-0e656d8c1143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:17:00.753340 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-0b5f5d66-5291-4f34-9a84-02c6757395ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af2f5f52ab5a45d7a38ba24071905110', 'request_id': 'req-0b5f5d66-5291-4f34-9a84-02c6757395ed'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:17:00.763307 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-0b5f5d66-5291-4f34-9a84-02c6757395ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af2f5f52ab5a45d7a38ba24071905110', 'request_id': 'req-0b5f5d66-5291-4f34-9a84-02c6757395ed'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:17:00.772330 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-0b5f5d66-5291-4f34-9a84-02c6757395ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af2f5f52ab5a45d7a38ba24071905110', 'request_id': 'req-0b5f5d66-5291-4f34-9a84-02c6757395ed'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:17:00.789882 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-0b5f5d66-5291-4f34-9a84-02c6757395ed admin admin] The token cache is being invalidate because project af2f5f52ab5a45d7a38ba24071905110 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81617) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:17:00.790305 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-0b5f5d66-5291-4f34-9a84-02c6757395ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:17:00.790895 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-0b5f5d66-5291-4f34-9a84-02c6757395ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:17:00.794417 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1085/2165] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:17:00 2022] DELETE /identity/v3/projects/af2f5f52ab5a45d7a38ba24071905110 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:17:00.806634 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa883a-f2fb-49ff-898b-25d1151b5248 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:17:00.807280 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa883a-f2fb-49ff-898b-25d1151b5248 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:17:00.807713 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa883a-f2fb-49ff-898b-25d1151b5248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:17:00.832933 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-5bfa883a-f2fb-49ff-898b-25d1151b5248 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:17:00.834355 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-5bfa883a-f2fb-49ff-898b-25d1151b5248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:17:00.956664 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dbadbc-9b1b-4522-8853-0e656d8c1143 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:17:00.986188 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1086/2166] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:17:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:17:00.991898 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f4dac-eeff-464f-868f-b84e502ff732 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:17:00.992367 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f4dac-eeff-464f-868f-b84e502ff732 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:17:00.992850 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f4dac-eeff-464f-868f-b84e502ff732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:17:01.043742 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-d31f4dac-eeff-464f-868f-b84e502ff732 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:17:01.047203 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-d31f4dac-eeff-464f-868f-b84e502ff732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:17:01.114293 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46433145-26a2-418c-a80e-7968ca4a1dbb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:17:01.114955 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46433145-26a2-418c-a80e-7968ca4a1dbb service cinder] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:17:01.115241 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46433145-26a2-418c-a80e-7968ca4a1dbb service cinder] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:17:01.115515 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46433145-26a2-418c-a80e-7968ca4a1dbb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:17:01.117499 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-46433145-26a2-418c-a80e-7968ca4a1dbb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:17:01.118857 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1087/2167] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:17:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:17:01.126006 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31f4dac-eeff-464f-868f-b84e502ff732 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:17:01.140704 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:17:01.161686 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31f4dac-eeff-464f-868f-b84e502ff732 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:17:01.175266 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b36060-be2e-4f94-b39c-4ca3eccab126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:17:01.176903 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b36060-be2e-4f94-b39c-4ca3eccab126 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:17:01.178221 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b36060-be2e-4f94-b39c-4ca3eccab126 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:17:01.179033 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b36060-be2e-4f94-b39c-4ca3eccab126 admin admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:17:01.180552 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b36060-be2e-4f94-b39c-4ca3eccab126 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:17:01.181620 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b36060-be2e-4f94-b39c-4ca3eccab126 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:17:01.185791 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b36060-be2e-4f94-b39c-4ca3eccab126 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:17:01.227812 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1088/2168] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:17:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:17:01.236633 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b36060-be2e-4f94-b39c-4ca3eccab126 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:17:01.276620 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b36060-be2e-4f94-b39c-4ca3eccab126 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:17:01.286265 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba51dd4-1ffa-47e9-ae46-b30e3012759d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:17:01.287167 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba51dd4-1ffa-47e9-ae46-b30e3012759d admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:17:01.287652 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba51dd4-1ffa-47e9-ae46-b30e3012759d admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:17:01.288117 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba51dd4-1ffa-47e9-ae46-b30e3012759d admin admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:17:01.289245 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba51dd4-1ffa-47e9-ae46-b30e3012759d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:17:01.291676 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46433145-26a2-418c-a80e-7968ca4a1dbb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:17:01.293047 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba51dd4-1ffa-47e9-ae46-b30e3012759d admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:17:01.300030 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46433145-26a2-418c-a80e-7968ca4a1dbb service cinder] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:17:01.303654 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:17:01.311883 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1081/2169] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:17:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 600 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:17:01.359209 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1089/2170] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:17:01 2022] POST /identity/v3/projects => generated 383 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:17:01.360934 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46433145-26a2-418c-a80e-7968ca4a1dbb service cinder] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:17:01.372626 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba51dd4-1ffa-47e9-ae46-b30e3012759d admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:17:01.381421 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-0ba51dd4-1ffa-47e9-ae46-b30e3012759d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:17:01.409542 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46433145-26a2-418c-a80e-7968ca4a1dbb service cinder] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:17:01.544003 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d190d-df20-4c71-b4c8-1e2c92d6c570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:17:01.544636 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d190d-df20-4c71-b4c8-1e2c92d6c570 admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:17:01.544920 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d190d-df20-4c71-b4c8-1e2c92d6c570 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:17:01.545198 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d190d-df20-4c71-b4c8-1e2c92d6c570 admin admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:17:01.546144 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1d190d-df20-4c71-b4c8-1e2c92d6c570 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:17:01.549276 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1d190d-df20-4c71-b4c8-1e2c92d6c570 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:17:01.589904 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1082/2171] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:17:01 2022] POST /identity/v3/users => generated 292 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:17:01.590212 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba51dd4-1ffa-47e9-ae46-b30e3012759d admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:17:01.591807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d190d-df20-4c71-b4c8-1e2c92d6c570 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:17:01.600857 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79751300-19c1-4582-93c8-4900bd14419a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:17:01.603132 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79751300-19c1-4582-93c8-4900bd14419a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:17:01.605772 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79751300-19c1-4582-93c8-4900bd14419a admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:17:01.606129 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79751300-19c1-4582-93c8-4900bd14419a admin admin] PATH_INFO: `/v3/users/c264d818ff5c466685cc85aa58a77f32` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:17:01.610297 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79751300-19c1-4582-93c8-4900bd14419a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c264d818ff5c466685cc85aa58a77f32)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:17:01.614020 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d190d-df20-4c71-b4c8-1e2c92d6c570 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:17:01.616964 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79751300-19c1-4582-93c8-4900bd14419a admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:17:01.632160 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3068b45c-1a52-49ff-b072-a8091522a030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:17:01.632778 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3068b45c-1a52-49ff-b072-a8091522a030 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:17:01.633452 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3068b45c-1a52-49ff-b072-a8091522a030 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:17:01.633686 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3068b45c-1a52-49ff-b072-a8091522a030 admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:17:01.635495 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3068b45c-1a52-49ff-b072-a8091522a030 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:17:01.643957 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3068b45c-1a52-49ff-b072-a8091522a030 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:17:01.694022 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1083/2172] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:17:01 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 16:17:01.696775 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3068b45c-1a52-49ff-b072-a8091522a030 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:17:01.746968 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-79751300-19c1-4582-93c8-4900bd14419a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c264d818ff5c466685cc85aa58a77f32'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:17:01.747785 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-79751300-19c1-4582-93c8-4900bd14419a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c264d818ff5c466685cc85aa58a77f32'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:17:01.752596 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3068b45c-1a52-49ff-b072-a8091522a030 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:17:01.789768 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-79751300-19c1-4582-93c8-4900bd14419a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c264d818ff5c466685cc85aa58a77f32'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:17:01.794161 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5149457-3a37-4292-9184-a77d31c688f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:17:01.795289 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5149457-3a37-4292-9184-a77d31c688f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:17:01.805767 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5149457-3a37-4292-9184-a77d31c688f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:17:01.805767 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5149457-3a37-4292-9184-a77d31c688f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:17:01.809573 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5149457-3a37-4292-9184-a77d31c688f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:17:01.813014 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5149457-3a37-4292-9184-a77d31c688f0 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:17:01.867105 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1084/2173] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:17:01 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:17:01.871231 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5149457-3a37-4292-9184-a77d31c688f0 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:17:01.873797 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-79751300-19c1-4582-93c8-4900bd14419a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c264d818ff5c466685cc85aa58a77f32'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:17:01.876046 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-79751300-19c1-4582-93c8-4900bd14419a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c264d818ff5c466685cc85aa58a77f32'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:17:01.881179 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-e5149457-3a37-4292-9184-a77d31c688f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:17:01.914449 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1090/2174] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:17:01 2022] DELETE /identity/v3/users/c264d818ff5c466685cc85aa58a77f32 => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:17:01.920228 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5224c8ca-e017-4b70-8d3c-752eac2ef1e6 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:17:01.926479 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5224c8ca-e017-4b70-8d3c-752eac2ef1e6 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:17:01.926479 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5224c8ca-e017-4b70-8d3c-752eac2ef1e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:17:01.944726 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5149457-3a37-4292-9184-a77d31c688f0 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:17:02.034851 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-5224c8ca-e017-4b70-8d3c-752eac2ef1e6 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:17:02.044821 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-5224c8ca-e017-4b70-8d3c-752eac2ef1e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:17:02.171653 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22afb0c-75b4-4213-94f7-46ad7be65179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:17:02.172570 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22afb0c-75b4-4213-94f7-46ad7be65179 admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:17:02.172949 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22afb0c-75b4-4213-94f7-46ad7be65179 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:17:02.173868 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22afb0c-75b4-4213-94f7-46ad7be65179 admin admin] PATH_INFO: `/v3/projects/027b06570b2a41c4a0b43f3529340077/users/960704dfe18c4d739e90d00288724983/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:17:02.197341 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22afb0c-75b4-4213-94f7-46ad7be65179 admin admin] RBAC: Authorizing `identity:create_grant(project_id=027b06570b2a41c4a0b43f3529340077, user_id=960704dfe18c4d739e90d00288724983, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:17:02.200765 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22afb0c-75b4-4213-94f7-46ad7be65179 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:17:02.264850 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:17:02.291385 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1085/2175] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:17:01 2022] PUT /identity/v3/projects/027b06570b2a41c4a0b43f3529340077/users/960704dfe18c4d739e90d00288724983/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:17:02.294268 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d095e6-a5d1-4c78-9ab5-e1a5693b3cdc None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:17:02.295015 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d095e6-a5d1-4c78-9ab5-e1a5693b3cdc None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:17:02.295250 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d095e6-a5d1-4c78-9ab5-e1a5693b3cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:17:02.395576 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-57d095e6-a5d1-4c78-9ab5-e1a5693b3cdc None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:17:02.402854 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-57d095e6-a5d1-4c78-9ab5-e1a5693b3cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:17:02.410840 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1091/2176] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:17:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 493 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:17:02.488667 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5224c8ca-e017-4b70-8d3c-752eac2ef1e6 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:17:02.490446 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-5224c8ca-e017-4b70-8d3c-752eac2ef1e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:17:02.517928 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:17:02.615954 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1086/2177] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:17:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:17:02.637648 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d095e6-a5d1-4c78-9ab5-e1a5693b3cdc None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:17:02.642668 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-57d095e6-a5d1-4c78-9ab5-e1a5693b3cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:17:02.711352 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5224c8ca-e017-4b70-8d3c-752eac2ef1e6 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:17:02.799980 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d095e6-a5d1-4c78-9ab5-e1a5693b3cdc None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:17:02.812166 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6817f6-ec3f-4b9b-8c27-59f9bcd2e64b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:17:02.812713 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6817f6-ec3f-4b9b-8c27-59f9bcd2e64b service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:17:02.812992 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6817f6-ec3f-4b9b-8c27-59f9bcd2e64b service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:17:02.813264 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6817f6-ec3f-4b9b-8c27-59f9bcd2e64b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:17:02.831194 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6817f6-ec3f-4b9b-8c27-59f9bcd2e64b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:17:02.834986 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6817f6-ec3f-4b9b-8c27-59f9bcd2e64b service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:17:02.841915 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b64eb0-0906-405a-9365-78eff8633c33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:17:02.842999 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b64eb0-0906-405a-9365-78eff8633c33 service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:17:02.843512 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b64eb0-0906-405a-9365-78eff8633c33 service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:17:02.845374 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b64eb0-0906-405a-9365-78eff8633c33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:17:02.856574 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1087/2178] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:17:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 16:17:02.877908 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b64eb0-0906-405a-9365-78eff8633c33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:17:02.880109 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b64eb0-0906-405a-9365-78eff8633c33 service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:17:02.903210 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1092/2179] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:17:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 16:17:04.995866 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6817f6-ec3f-4b9b-8c27-59f9bcd2e64b service neutron] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:17:04.997473 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-df6817f6-ec3f-4b9b-8c27-59f9bcd2e64b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:17:05.018959 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6817f6-ec3f-4b9b-8c27-59f9bcd2e64b service neutron] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:17:05.025192 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4989a2-d65f-4a7c-a371-37dba93dd788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:17:05.025830 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4989a2-d65f-4a7c-a371-37dba93dd788 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:17:05.026117 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4989a2-d65f-4a7c-a371-37dba93dd788 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:17:05.026439 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4989a2-d65f-4a7c-a371-37dba93dd788 admin admin] PATH_INFO: `/v3/projects/5fcd58a1bf714f9fb9a5a802ae721596` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:17:05.028239 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4989a2-d65f-4a7c-a371-37dba93dd788 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fcd58a1bf714f9fb9a5a802ae721596)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:17:05.030605 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4989a2-d65f-4a7c-a371-37dba93dd788 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:17:05.176186 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-ed4989a2-d65f-4a7c-a371-37dba93dd788 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fcd58a1bf714f9fb9a5a802ae721596', 'request_id': 'req-ed4989a2-d65f-4a7c-a371-37dba93dd788'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:17:05.193656 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-ed4989a2-d65f-4a7c-a371-37dba93dd788 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fcd58a1bf714f9fb9a5a802ae721596', 'request_id': 'req-ed4989a2-d65f-4a7c-a371-37dba93dd788'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:17:05.227295 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-ed4989a2-d65f-4a7c-a371-37dba93dd788 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fcd58a1bf714f9fb9a5a802ae721596', 'request_id': 'req-ed4989a2-d65f-4a7c-a371-37dba93dd788'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:17:05.271727 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-ed4989a2-d65f-4a7c-a371-37dba93dd788 admin admin] The token cache is being invalidate because project 5fcd58a1bf714f9fb9a5a802ae721596 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81618) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:17:05.271727 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-ed4989a2-d65f-4a7c-a371-37dba93dd788 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:17:05.272186 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-ed4989a2-d65f-4a7c-a371-37dba93dd788 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:17:05.276808 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1088/2180] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:17:04 2022] DELETE /identity/v3/projects/5fcd58a1bf714f9fb9a5a802ae721596 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:17:05.283881 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b118edac-087b-4f38-b616-d0b3b863b548 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:17:05.284396 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b118edac-087b-4f38-b616-d0b3b863b548 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:17:05.284631 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b118edac-087b-4f38-b616-d0b3b863b548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:17:05.324278 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-b118edac-087b-4f38-b616-d0b3b863b548 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:17:05.327351 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-b118edac-087b-4f38-b616-d0b3b863b548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:17:05.453940 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1093/2181] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:17:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:17:05.467365 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f11aa-1260-43a3-851f-7f27e2f8fa72 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:17:05.467795 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f11aa-1260-43a3-851f-7f27e2f8fa72 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:17:05.468178 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f11aa-1260-43a3-851f-7f27e2f8fa72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:17:05.509831 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-e37f11aa-1260-43a3-851f-7f27e2f8fa72 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:17:05.511495 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-e37f11aa-1260-43a3-851f-7f27e2f8fa72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:17:05.523952 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1089/2182] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:17:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:17:05.529906 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b118edac-087b-4f38-b616-d0b3b863b548 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:17:05.547884 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b118edac-087b-4f38-b616-d0b3b863b548 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:17:05.571210 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cca55a-da23-45a6-8f1a-ed672407c5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:17:05.572904 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cca55a-da23-45a6-8f1a-ed672407c5b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:17:05.573185 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cca55a-da23-45a6-8f1a-ed672407c5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:17:05.578021 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cca55a-da23-45a6-8f1a-ed672407c5b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:17:05.578021 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cca55a-da23-45a6-8f1a-ed672407c5b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:17:05.578021 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cca55a-da23-45a6-8f1a-ed672407c5b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:17:05.585940 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cca55a-da23-45a6-8f1a-ed672407c5b4 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:17:05.621897 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37f11aa-1260-43a3-851f-7f27e2f8fa72 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:17:05.634044 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1094/2183] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:17:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:17:05.640101 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37f11aa-1260-43a3-851f-7f27e2f8fa72 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:17:05.646423 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0de581-1b86-494b-8f6a-a087be48406d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:17:05.647958 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0de581-1b86-494b-8f6a-a087be48406d admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:17:05.648439 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0de581-1b86-494b-8f6a-a087be48406d admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:17:05.648905 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0de581-1b86-494b-8f6a-a087be48406d admin admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:17:05.649792 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0de581-1b86-494b-8f6a-a087be48406d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:17:05.653138 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0de581-1b86-494b-8f6a-a087be48406d admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:17:05.678987 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:17:05.712096 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cca55a-da23-45a6-8f1a-ed672407c5b4 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:17:05.722704 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1090/2184] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:17:05 2022] POST /identity/v3/projects => generated 383 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:17:05.730071 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cca55a-da23-45a6-8f1a-ed672407c5b4 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:17:05.841612 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:17:05.880720 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53844f5c-eec1-450d-b4bb-a6aa70005879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:17:05.881496 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53844f5c-eec1-450d-b4bb-a6aa70005879 admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:17:05.881778 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53844f5c-eec1-450d-b4bb-a6aa70005879 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:17:05.882801 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53844f5c-eec1-450d-b4bb-a6aa70005879 admin admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:17:05.886628 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53844f5c-eec1-450d-b4bb-a6aa70005879 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:17:05.896066 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53844f5c-eec1-450d-b4bb-a6aa70005879 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:17:05.946553 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1095/2185] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:17:05 2022] POST /identity/v3/users => generated 292 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:17:05.951834 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0de581-1b86-494b-8f6a-a087be48406d admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:17:05.986943 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0de581-1b86-494b-8f6a-a087be48406d admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:17:06.006417 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a963f3c-4654-4970-b8c9-9191dc7da663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:17:06.007504 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a963f3c-4654-4970-b8c9-9191dc7da663 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:17:06.007504 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a963f3c-4654-4970-b8c9-9191dc7da663 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:17:06.007835 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a963f3c-4654-4970-b8c9-9191dc7da663 admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:17:06.008538 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a963f3c-4654-4970-b8c9-9191dc7da663 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:17:06.013191 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a963f3c-4654-4970-b8c9-9191dc7da663 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:17:06.043668 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1091/2186] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:17:05 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:17:06.043972 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53844f5c-eec1-450d-b4bb-a6aa70005879 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:17:06.062426 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53844f5c-eec1-450d-b4bb-a6aa70005879 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:17:06.075774 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02bfa85-7f74-444a-9ba5-a2cd64fcff5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:17:06.076573 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02bfa85-7f74-444a-9ba5-a2cd64fcff5b admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:17:06.076733 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02bfa85-7f74-444a-9ba5-a2cd64fcff5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:17:06.077104 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02bfa85-7f74-444a-9ba5-a2cd64fcff5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:17:06.077601 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02bfa85-7f74-444a-9ba5-a2cd64fcff5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:17:06.080293 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02bfa85-7f74-444a-9ba5-a2cd64fcff5b admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:17:06.109292 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1096/2187] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:17:06 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 16:17:06.118655 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a963f3c-4654-4970-b8c9-9191dc7da663 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:17:06.132646 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a963f3c-4654-4970-b8c9-9191dc7da663 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:17:06.141956 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74241d1-2bde-4386-aefe-e51641530d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:17:06.142686 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74241d1-2bde-4386-aefe-e51641530d70 admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:17:06.144065 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74241d1-2bde-4386-aefe-e51641530d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:17:06.144905 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74241d1-2bde-4386-aefe-e51641530d70 admin admin] PATH_INFO: `/v3/projects/5fc6461a081c479fa20c9fb15a2b1575/users/e1c0b2cfaa9f485f9116e012c62120fb/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:17:06.176922 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74241d1-2bde-4386-aefe-e51641530d70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fc6461a081c479fa20c9fb15a2b1575, user_id=e1c0b2cfaa9f485f9116e012c62120fb, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:17:06.182467 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74241d1-2bde-4386-aefe-e51641530d70 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:17:06.231216 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1092/2188] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:17:06 2022] PUT /identity/v3/projects/5fc6461a081c479fa20c9fb15a2b1575/users/e1c0b2cfaa9f485f9116e012c62120fb/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:17:06.241062 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71656166-e210-4998-b7d5-0c670b5ead8b None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:17:06.241556 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71656166-e210-4998-b7d5-0c670b5ead8b None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:17:06.241977 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71656166-e210-4998-b7d5-0c670b5ead8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:17:06.301992 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-71656166-e210-4998-b7d5-0c670b5ead8b None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:17:06.305607 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-71656166-e210-4998-b7d5-0c670b5ead8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:17:06.449137 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1097/2189] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:17:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:17:06.460084 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74241d1-2bde-4386-aefe-e51641530d70 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:17:06.461878 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-b74241d1-2bde-4386-aefe-e51641530d70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:17:06.619448 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74241d1-2bde-4386-aefe-e51641530d70 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:17:06.632161 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d98b1f8-986b-461d-84a6-da4784667ae1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:17:06.633748 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d98b1f8-986b-461d-84a6-da4784667ae1 service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:17:06.634206 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d98b1f8-986b-461d-84a6-da4784667ae1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:17:06.635086 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d98b1f8-986b-461d-84a6-da4784667ae1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:17:06.662000 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d98b1f8-986b-461d-84a6-da4784667ae1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:17:06.669759 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d98b1f8-986b-461d-84a6-da4784667ae1 service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:17:06.701199 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1093/2190] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:17:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:17:08.704825 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff0c5c4-25b2-49a0-9ea0-6bbdd949da6e None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:17:08.706740 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff0c5c4-25b2-49a0-9ea0-6bbdd949da6e None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:17:08.707763 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff0c5c4-25b2-49a0-9ea0-6bbdd949da6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:17:08.830106 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-3ff0c5c4-25b2-49a0-9ea0-6bbdd949da6e None None] MFA Rules not processed for user `e1c0b2cfaa9f485f9116e012c62120fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:17:08.832510 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-3ff0c5c4-25b2-49a0-9ea0-6bbdd949da6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:17:09.099023 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1098/2191] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:17:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 397 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:17:09.104555 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d98b1f8-986b-461d-84a6-da4784667ae1 service neutron] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:17:09.106924 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-1d98b1f8-986b-461d-84a6-da4784667ae1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:17:09.398509 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d98b1f8-986b-461d-84a6-da4784667ae1 service neutron] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:17:09.416311 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f92786b-eefa-4306-bbfe-dcfdb8af927a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:17:09.417420 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f92786b-eefa-4306-bbfe-dcfdb8af927a service cinder] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:17:09.418053 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f92786b-eefa-4306-bbfe-dcfdb8af927a service cinder] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:17:09.418589 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f92786b-eefa-4306-bbfe-dcfdb8af927a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:17:09.452799 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f92786b-eefa-4306-bbfe-dcfdb8af927a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:17:09.456501 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f92786b-eefa-4306-bbfe-dcfdb8af927a service cinder] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:17:09.508140 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1094/2192] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:17:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:17:11.402819 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a601bd02-7a44-4fa3-8aa5-ee0e85a6b648 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:17:11.403445 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a601bd02-7a44-4fa3-8aa5-ee0e85a6b648 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:17:11.403445 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a601bd02-7a44-4fa3-8aa5-ee0e85a6b648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:17:11.442831 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-a601bd02-7a44-4fa3-8aa5-ee0e85a6b648 None None] MFA Rules not processed for user `e1c0b2cfaa9f485f9116e012c62120fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:17:11.446611 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-a601bd02-7a44-4fa3-8aa5-ee0e85a6b648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:17:11.468499 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1099/2193] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:17:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:17:13.895354 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b19b31-8044-4497-b78f-541dff56fae3 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:17:13.896509 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b19b31-8044-4497-b78f-541dff56fae3 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:17:13.897153 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b19b31-8044-4497-b78f-541dff56fae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:17:13.941778 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-84b19b31-8044-4497-b78f-541dff56fae3 None None] MFA Rules not processed for user `960704dfe18c4d739e90d00288724983`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:17:13.943504 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-84b19b31-8044-4497-b78f-541dff56fae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:17:14.036556 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:17:14.062416 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1095/2194] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:17:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:17:14.072077 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a601bd02-7a44-4fa3-8aa5-ee0e85a6b648 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:17:14.074327 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-a601bd02-7a44-4fa3-8aa5-ee0e85a6b648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:17:14.106596 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a601bd02-7a44-4fa3-8aa5-ee0e85a6b648 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:17:14.113434 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05c339d-0535-4bff-9b63-912664648e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:17:14.114399 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c339d-0535-4bff-9b63-912664648e34 admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:17:14.114979 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c339d-0535-4bff-9b63-912664648e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:17:14.115443 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c339d-0535-4bff-9b63-912664648e34 admin admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:17:14.116234 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05c339d-0535-4bff-9b63-912664648e34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:17:14.119499 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05c339d-0535-4bff-9b63-912664648e34 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:17:14.180926 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1100/2195] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:17:14 2022] POST /identity/v3/projects => generated 385 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:17:14.186966 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b19b31-8044-4497-b78f-541dff56fae3 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:17:14.211777 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b19b31-8044-4497-b78f-541dff56fae3 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:17:14.317173 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde5cad1-9532-45ab-8e89-0470bd0a34b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:17:14.318613 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde5cad1-9532-45ab-8e89-0470bd0a34b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:17:14.319226 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde5cad1-9532-45ab-8e89-0470bd0a34b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:17:14.319902 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde5cad1-9532-45ab-8e89-0470bd0a34b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:17:14.321250 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde5cad1-9532-45ab-8e89-0470bd0a34b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:17:14.324921 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde5cad1-9532-45ab-8e89-0470bd0a34b6 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:17:14.366812 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1096/2196] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:17:14 2022] POST /identity/v3/users => generated 299 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:17:14.368135 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05c339d-0535-4bff-9b63-912664648e34 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:17:14.411596 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05c339d-0535-4bff-9b63-912664648e34 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:17:14.420942 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4286f8dc-8b4c-4cb2-8f11-c1bc889baaff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:17:14.421856 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4286f8dc-8b4c-4cb2-8f11-c1bc889baaff admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:17:14.422198 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4286f8dc-8b4c-4cb2-8f11-c1bc889baaff admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:17:14.423224 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4286f8dc-8b4c-4cb2-8f11-c1bc889baaff admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:17:14.424329 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4286f8dc-8b4c-4cb2-8f11-c1bc889baaff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:17:14.427436 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4286f8dc-8b4c-4cb2-8f11-c1bc889baaff admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:17:14.453387 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1101/2197] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:17:14 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:17:14.457899 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde5cad1-9532-45ab-8e89-0470bd0a34b6 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:17:14.477581 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde5cad1-9532-45ab-8e89-0470bd0a34b6 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:17:14.486926 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a2d651-ba6b-42cb-ae07-970f87b2e4ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:17:14.487925 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2d651-ba6b-42cb-ae07-970f87b2e4ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:17:14.488333 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2d651-ba6b-42cb-ae07-970f87b2e4ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:17:14.488794 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2d651-ba6b-42cb-ae07-970f87b2e4ff admin admin] PATH_INFO: `/v3/projects/ecbaa32c295d45cb83db075c5e180c30/users/edff2b6344e34f1b877d0c908c43290f/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:17:14.513896 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a2d651-ba6b-42cb-ae07-970f87b2e4ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecbaa32c295d45cb83db075c5e180c30, user_id=edff2b6344e34f1b877d0c908c43290f, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:17:14.518974 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a2d651-ba6b-42cb-ae07-970f87b2e4ff admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:17:14.541521 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1097/2198] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:17:14 2022] PUT /identity/v3/projects/ecbaa32c295d45cb83db075c5e180c30/users/edff2b6344e34f1b877d0c908c43290f/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:17:14.546927 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4286f8dc-8b4c-4cb2-8f11-c1bc889baaff admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:17:14.563087 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4286f8dc-8b4c-4cb2-8f11-c1bc889baaff admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:17:14.597437 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:17:14.658892 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9b39d5-a6c3-496b-8618-1c3948ccd53a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:17:14.659523 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9b39d5-a6c3-496b-8618-1c3948ccd53a admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:17:14.659807 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9b39d5-a6c3-496b-8618-1c3948ccd53a admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:17:14.660100 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9b39d5-a6c3-496b-8618-1c3948ccd53a admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:17:14.660750 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9b39d5-a6c3-496b-8618-1c3948ccd53a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:17:14.662580 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9b39d5-a6c3-496b-8618-1c3948ccd53a admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:17:14.674561 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1102/2199] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:17:14 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:17:14.679985 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a2d651-ba6b-42cb-ae07-970f87b2e4ff admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:17:14.706722 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a2d651-ba6b-42cb-ae07-970f87b2e4ff admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:17:14.719916 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba316be-310c-48d4-9455-63649632d9e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:17:14.721396 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba316be-310c-48d4-9455-63649632d9e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:17:14.722091 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba316be-310c-48d4-9455-63649632d9e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:17:14.722701 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba316be-310c-48d4-9455-63649632d9e5 admin admin] PATH_INFO: `/v3/projects/ecbaa32c295d45cb83db075c5e180c30/users/edff2b6344e34f1b877d0c908c43290f/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:17:14.727098 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba316be-310c-48d4-9455-63649632d9e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecbaa32c295d45cb83db075c5e180c30, user_id=edff2b6344e34f1b877d0c908c43290f, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:17:14.733043 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba316be-310c-48d4-9455-63649632d9e5 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:17:14.761387 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1098/2200] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:17:14 2022] PUT /identity/v3/projects/ecbaa32c295d45cb83db075c5e180c30/users/edff2b6344e34f1b877d0c908c43290f/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 16:17:19.975892 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9482674c-0254-4d76-9fee-8d745ca048c4 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:17:19.976713 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9482674c-0254-4d76-9fee-8d745ca048c4 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:17:19.976713 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9482674c-0254-4d76-9fee-8d745ca048c4 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:17:19.978630 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1103/2201] 10.222.0.57 () {54 vars in 921 bytes} [Mon Sep 19 16:17:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:17:19.989849 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d61e2d7-ca69-49c6-86a1-3ac9d64e75e8 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:17:19.991988 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d61e2d7-ca69-49c6-86a1-3ac9d64e75e8 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:17:19.993697 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d61e2d7-ca69-49c6-86a1-3ac9d64e75e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:17:20.052300 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-5d61e2d7-ca69-49c6-86a1-3ac9d64e75e8 None None] MFA Rules not processed for user `1ecd01cc72834b72b5ff2c4b47249737`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:17:20.056730 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-5d61e2d7-ca69-49c6-86a1-3ac9d64e75e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:17:20.252866 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1099/2202] 10.222.0.57 () {58 vars in 1018 bytes} [Mon Sep 19 16:17:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:17:20.290941 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9482674c-0254-4d76-9fee-8d745ca048c4 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:17:20.292941 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-9482674c-0254-4d76-9fee-8d745ca048c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:17:20.449699 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9482674c-0254-4d76-9fee-8d745ca048c4 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:17:20.455754 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310a2ca0-4b43-4aa3-8faa-fd48a9b450cc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5632ade0cc6f4ae28f72160de68486cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:17:20.456585 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a2ca0-4b43-4aa3-8faa-fd48a9b450cc service glance] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:17:20.457555 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a2ca0-4b43-4aa3-8faa-fd48a9b450cc service glance] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:17:20.457844 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a2ca0-4b43-4aa3-8faa-fd48a9b450cc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:17:20.490808 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310a2ca0-4b43-4aa3-8faa-fd48a9b450cc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:17:20.494172 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310a2ca0-4b43-4aa3-8faa-fd48a9b450cc service glance] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:17:20.554992 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1104/2203] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:17:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:17:20.673643 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2717eda5-3f89-49b6-b381-7fdbbc6c7006 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:17:20.673759 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2717eda5-3f89-49b6-b381-7fdbbc6c7006 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:17:20.673991 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2717eda5-3f89-49b6-b381-7fdbbc6c7006 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:17:20.675427 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1100/2204] 10.222.0.57 () {54 vars in 921 bytes} [Mon Sep 19 16:17:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:17:20.688729 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e922a-38cb-4a5e-8c1d-b11dc2d6696d None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:17:20.689027 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e922a-38cb-4a5e-8c1d-b11dc2d6696d None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:17:20.689279 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e922a-38cb-4a5e-8c1d-b11dc2d6696d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:17:20.730553 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-dc6e922a-38cb-4a5e-8c1d-b11dc2d6696d None None] MFA Rules not processed for user `50355d7b1c6c4f0bbbbc7fe56efb9039`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:17:20.731949 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-dc6e922a-38cb-4a5e-8c1d-b11dc2d6696d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:17:20.857347 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1105/2205] 10.222.0.57 () {58 vars in 1018 bytes} [Mon Sep 19 16:17:20 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:17:20.883327 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2717eda5-3f89-49b6-b381-7fdbbc6c7006 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:17:20.887043 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-2717eda5-3f89-49b6-b381-7fdbbc6c7006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:17:20.943474 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2717eda5-3f89-49b6-b381-7fdbbc6c7006 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:17:20.966526 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2543c68b-591f-4ad5-9efe-dc547f771f89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:17:20.967611 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2543c68b-591f-4ad5-9efe-dc547f771f89 service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:17:20.968098 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2543c68b-591f-4ad5-9efe-dc547f771f89 service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:17:20.968614 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2543c68b-591f-4ad5-9efe-dc547f771f89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:17:21.004256 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2543c68b-591f-4ad5-9efe-dc547f771f89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:17:21.005730 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2543c68b-591f-4ad5-9efe-dc547f771f89 service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:17:21.033927 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1101/2206] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:17:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:17:21.433145 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f068f91-9355-4a10-85b9-2a6f5bc1a31e None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:17:21.435004 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f068f91-9355-4a10-85b9-2a6f5bc1a31e None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:17:21.436085 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f068f91-9355-4a10-85b9-2a6f5bc1a31e None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:17:21.438236 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1106/2207] 10.222.0.57 () {54 vars in 921 bytes} [Mon Sep 19 16:17:21 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 16:17:21.450963 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103eb1a-9710-472b-915a-70fd3d616ebe None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:17:21.451694 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103eb1a-9710-472b-915a-70fd3d616ebe None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:17:21.452129 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103eb1a-9710-472b-915a-70fd3d616ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:17:21.510800 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-8103eb1a-9710-472b-915a-70fd3d616ebe None None] MFA Rules not processed for user `1ecd01cc72834b72b5ff2c4b47249737`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:17:21.513494 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-8103eb1a-9710-472b-915a-70fd3d616ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:17:21.542679 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1102/2208] 10.222.0.57 () {58 vars in 1018 bytes} [Mon Sep 19 16:17:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:17:21.561963 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f068f91-9355-4a10-85b9-2a6f5bc1a31e None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:17:21.615732 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f068f91-9355-4a10-85b9-2a6f5bc1a31e None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:17:21.637644 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae7727b-6f92-4bc8-99a4-839e11bc2920 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:17:21.649943 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae7727b-6f92-4bc8-99a4-839e11bc2920 service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:17:21.650561 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae7727b-6f92-4bc8-99a4-839e11bc2920 service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:17:21.653668 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae7727b-6f92-4bc8-99a4-839e11bc2920 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:17:21.692204 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae7727b-6f92-4bc8-99a4-839e11bc2920 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:17:21.695041 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae7727b-6f92-4bc8-99a4-839e11bc2920 service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:17:21.712645 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1107/2209] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:17:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:17:23.539733 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6522dbc2-e0f6-4abd-b803-db0940b89bdf None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:17:23.540287 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6522dbc2-e0f6-4abd-b803-db0940b89bdf None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:17:23.540513 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6522dbc2-e0f6-4abd-b803-db0940b89bdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:17:23.595499 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-6522dbc2-e0f6-4abd-b803-db0940b89bdf None None] MFA Rules not processed for user `edff2b6344e34f1b877d0c908c43290f`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:17:23.597057 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-6522dbc2-e0f6-4abd-b803-db0940b89bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:17:23.609094 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:17:23.763607 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1103/2210] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:17:23 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:17:23.787506 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae7727b-6f92-4bc8-99a4-839e11bc2920 service neutron] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:17:23.812842 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae7727b-6f92-4bc8-99a4-839e11bc2920 service neutron] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:17:23.820246 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da95b464-bb64-453b-a2fa-6f848ef0e954 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5632ade0cc6f4ae28f72160de68486cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:17:23.820867 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da95b464-bb64-453b-a2fa-6f848ef0e954 service glance] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:17:23.821173 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da95b464-bb64-453b-a2fa-6f848ef0e954 service glance] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:17:23.821390 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da95b464-bb64-453b-a2fa-6f848ef0e954 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:17:24.014574 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:17:24.017679 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da95b464-bb64-453b-a2fa-6f848ef0e954 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:17:24.019372 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da95b464-bb64-453b-a2fa-6f848ef0e954 service glance] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:17:24.031588 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1108/2211] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:17:23 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:17:24.385928 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac00e25-f832-4761-8482-9b51c009cffb None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:17:24.386449 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac00e25-f832-4761-8482-9b51c009cffb None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:17:24.387000 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac00e25-f832-4761-8482-9b51c009cffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:17:24.476302 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-0ac00e25-f832-4761-8482-9b51c009cffb None None] MFA Rules not processed for user `7222bd3b313b47e5a4fa0ebc9ff0cdd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:17:24.478610 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-0ac00e25-f832-4761-8482-9b51c009cffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:17:24.720640 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1104/2212] 10.222.0.57 () {58 vars in 1009 bytes} [Mon Sep 19 16:17:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:17:24.746715 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da95b464-bb64-453b-a2fa-6f848ef0e954 service glance] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:17:24.748370 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-da95b464-bb64-453b-a2fa-6f848ef0e954 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:17:24.879564 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da95b464-bb64-453b-a2fa-6f848ef0e954 service glance] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:17:24.888429 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb90291-1c42-4b65-8099-c09a0fa26588 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '884a0524aca744b99c7496c11c3291ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:17:24.890169 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb90291-1c42-4b65-8099-c09a0fa26588 service swift] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:17:24.890982 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb90291-1c42-4b65-8099-c09a0fa26588 service swift] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:17:24.891930 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb90291-1c42-4b65-8099-c09a0fa26588 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:17:24.931272 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb90291-1c42-4b65-8099-c09a0fa26588 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:17:24.933664 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb90291-1c42-4b65-8099-c09a0fa26588 service swift] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:17:24.957874 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1109/2213] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Sep 19 16:17:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 19 16:17:28.286798 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e4d695-4189-45e3-8c68-39ab60b1a5cd None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:17:28.287815 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e4d695-4189-45e3-8c68-39ab60b1a5cd None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:17:28.288278 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e4d695-4189-45e3-8c68-39ab60b1a5cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:17:28.381502 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-f8e4d695-4189-45e3-8c68-39ab60b1a5cd None None] MFA Rules not processed for user `7222bd3b313b47e5a4fa0ebc9ff0cdd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:17:28.381502 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-f8e4d695-4189-45e3-8c68-39ab60b1a5cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:17:28.405226 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1105/2214] 10.222.0.57 () {58 vars in 1009 bytes} [Mon Sep 19 16:17:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:17:28.617874 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb90291-1c42-4b65-8099-c09a0fa26588 service swift] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:17:28.669363 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb90291-1c42-4b65-8099-c09a0fa26588 service swift] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:17:28.731502 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4676ff50-b6d8-4f74-b0a7-07ec27cbf84c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '884a0524aca744b99c7496c11c3291ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:17:28.740486 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4676ff50-b6d8-4f74-b0a7-07ec27cbf84c service swift] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:17:28.747720 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4676ff50-b6d8-4f74-b0a7-07ec27cbf84c service swift] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:17:28.748514 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4676ff50-b6d8-4f74-b0a7-07ec27cbf84c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:17:28.826625 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4676ff50-b6d8-4f74-b0a7-07ec27cbf84c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:17:28.830080 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4676ff50-b6d8-4f74-b0a7-07ec27cbf84c service swift] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:17:28.873523 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1110/2215] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Sep 19 16:17:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 16:17:38.342059 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=53 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:17:38.654451 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6445a1-b673-40aa-9573-caad2a22c10c None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:17:38.655159 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6445a1-b673-40aa-9573-caad2a22c10c None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:17:38.655597 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6445a1-b673-40aa-9573-caad2a22c10c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:17:38.702974 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-cb6445a1-b673-40aa-9573-caad2a22c10c None None] MFA Rules not processed for user `960704dfe18c4d739e90d00288724983`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:17:38.705118 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-cb6445a1-b673-40aa-9573-caad2a22c10c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:17:38.720393 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1106/2216] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:17:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:17:38.857320 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=40 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:17:45.651262 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f75da30-0541-4486-a17c-bda1e203c785 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:17:45.652327 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f75da30-0541-4486-a17c-bda1e203c785 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:17:45.652327 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f75da30-0541-4486-a17c-bda1e203c785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:17:45.694108 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-0f75da30-0541-4486-a17c-bda1e203c785 None None] MFA Rules not processed for user `7222bd3b313b47e5a4fa0ebc9ff0cdd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:17:45.695855 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-0f75da30-0541-4486-a17c-bda1e203c785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:17:45.721890 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1111/2217] 10.222.0.57 () {58 vars in 1009 bytes} [Mon Sep 19 16:17:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:17:45.745529 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6445a1-b673-40aa-9573-caad2a22c10c None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:17:45.789342 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6445a1-b673-40aa-9573-caad2a22c10c None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:17:45.798801 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec441c99-2362-47f7-a709-daf51f50e467 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '884a0524aca744b99c7496c11c3291ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:17:45.800279 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec441c99-2362-47f7-a709-daf51f50e467 service swift] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:17:45.801025 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec441c99-2362-47f7-a709-daf51f50e467 service swift] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:17:45.801754 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec441c99-2362-47f7-a709-daf51f50e467 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:17:45.818094 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec441c99-2362-47f7-a709-daf51f50e467 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:17:45.820721 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec441c99-2362-47f7-a709-daf51f50e467 service swift] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:17:45.832397 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1107/2218] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Sep 19 16:17:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 16:17:55.680754 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=3 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:17:55.834288 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=9 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:18:06.435573 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583a2406-bfc4-4f10-87ee-9dfde30c39dd None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:18:06.436415 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583a2406-bfc4-4f10-87ee-9dfde30c39dd None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:18:06.436673 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583a2406-bfc4-4f10-87ee-9dfde30c39dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:18:06.471157 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-583a2406-bfc4-4f10-87ee-9dfde30c39dd None None] MFA Rules not processed for user `7222bd3b313b47e5a4fa0ebc9ff0cdd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:18:06.473760 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-583a2406-bfc4-4f10-87ee-9dfde30c39dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:18:06.484082 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1112/2219] 10.222.0.57 () {58 vars in 967 bytes} [Mon Sep 19 16:18:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:18:06.520461 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec441c99-2362-47f7-a709-daf51f50e467 service swift] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:18:06.533796 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec441c99-2362-47f7-a709-daf51f50e467 service swift] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:18:06.539650 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb292e3a-157e-474e-9d5e-714144d8edcd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '884a0524aca744b99c7496c11c3291ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:18:06.540366 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb292e3a-157e-474e-9d5e-714144d8edcd service swift] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:18:06.540657 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb292e3a-157e-474e-9d5e-714144d8edcd service swift] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:18:06.540930 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb292e3a-157e-474e-9d5e-714144d8edcd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:18:06.554087 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb292e3a-157e-474e-9d5e-714144d8edcd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:18:06.557275 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb292e3a-157e-474e-9d5e-714144d8edcd service swift] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:18:06.569350 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1108/2220] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Sep 19 16:18:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 19 16:18:16.463690 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=3 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:18:16.564035 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=6 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:18:24.057714 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7c549-3653-4215-8033-2bfc90e68a6d None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:18:24.058587 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7c549-3653-4215-8033-2bfc90e68a6d None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:18:24.058810 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7c549-3653-4215-8033-2bfc90e68a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:18:24.081788 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-21f7c549-3653-4215-8033-2bfc90e68a6d None None] MFA Rules not processed for user `fd36089b3c2141218660ab2458376798`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:18:24.083228 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-21f7c549-3653-4215-8033-2bfc90e68a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:18:24.149724 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1113/2221] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:18:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:18:34.141973 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=25 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:18:38.519579 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb292e3a-157e-474e-9d5e-714144d8edcd service swift] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:18:38.521961 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-cb292e3a-157e-474e-9d5e-714144d8edcd service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:18:38.543268 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb292e3a-157e-474e-9d5e-714144d8edcd service swift] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:18:38.551117 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bd686d-9d1b-4cd2-b48f-bc07b6860a53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:18:38.551791 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd686d-9d1b-4cd2-b48f-bc07b6860a53 service nova] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:18:38.552113 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd686d-9d1b-4cd2-b48f-bc07b6860a53 service nova] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:18:38.552431 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd686d-9d1b-4cd2-b48f-bc07b6860a53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:18:38.555048 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-91bd686d-9d1b-4cd2-b48f-bc07b6860a53 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:18:38.568223 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bd686d-9d1b-4cd2-b48f-bc07b6860a53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:18:38.570712 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bd686d-9d1b-4cd2-b48f-bc07b6860a53 service nova] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:18:38.581171 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1109/2222] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:18:38 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:18:47.724857 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f7c549-3653-4215-8033-2bfc90e68a6d None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:18:47.736014 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f7c549-3653-4215-8033-2bfc90e68a6d None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:18:47.740093 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a041d3-6cb7-4fe6-9937-3686084e10b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:18:47.740851 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a041d3-6cb7-4fe6-9937-3686084e10b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:18:47.741151 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a041d3-6cb7-4fe6-9937-3686084e10b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:18:47.741441 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a041d3-6cb7-4fe6-9937-3686084e10b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:18:47.743331 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-f6a041d3-6cb7-4fe6-9937-3686084e10b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:18:47.757363 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bd686d-9d1b-4cd2-b48f-bc07b6860a53 service nova] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:18:47.759713 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-91bd686d-9d1b-4cd2-b48f-bc07b6860a53 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:18:47.760256 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a041d3-6cb7-4fe6-9937-3686084e10b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:18:47.763388 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a041d3-6cb7-4fe6-9937-3686084e10b0 service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:18:47.783730 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bd686d-9d1b-4cd2-b48f-bc07b6860a53 service nova] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:18:47.784906 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1114/2223] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:18:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:18:47.790874 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a135a113-1dfb-4800-80f5-3ed6a64fc6f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:18:47.791574 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a135a113-1dfb-4800-80f5-3ed6a64fc6f9 service nova] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:18:47.791914 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a135a113-1dfb-4800-80f5-3ed6a64fc6f9 service nova] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:18:47.792247 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a135a113-1dfb-4800-80f5-3ed6a64fc6f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:18:47.794607 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-a135a113-1dfb-4800-80f5-3ed6a64fc6f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:18:47.884320 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a135a113-1dfb-4800-80f5-3ed6a64fc6f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:18:47.886133 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a135a113-1dfb-4800-80f5-3ed6a64fc6f9 service nova] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:18:47.897203 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1110/2224] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:18:47 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 16:18:47.956058 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a041d3-6cb7-4fe6-9937-3686084e10b0 service neutron] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:18:47.965915 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a041d3-6cb7-4fe6-9937-3686084e10b0 service neutron] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:18:47.970860 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ce11a9-de95-4595-888b-a4c5ad88a62f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:18:47.971532 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ce11a9-de95-4595-888b-a4c5ad88a62f service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:18:47.971849 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ce11a9-de95-4595-888b-a4c5ad88a62f service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:18:47.972164 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ce11a9-de95-4595-888b-a4c5ad88a62f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:18:47.974253 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-e8ce11a9-de95-4595-888b-a4c5ad88a62f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:18:47.990073 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ce11a9-de95-4595-888b-a4c5ad88a62f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:18:47.991662 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ce11a9-de95-4595-888b-a4c5ad88a62f service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:18:48.004531 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1115/2225] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:18:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:18:51.244847 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a135a113-1dfb-4800-80f5-3ed6a64fc6f9 service nova] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:18:51.259202 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a135a113-1dfb-4800-80f5-3ed6a64fc6f9 service nova] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:18:51.265895 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fff23c5-45cc-4717-a004-59e6dc25f7d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:18:51.266631 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff23c5-45cc-4717-a004-59e6dc25f7d6 service nova] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:18:51.267038 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff23c5-45cc-4717-a004-59e6dc25f7d6 service nova] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:18:51.267375 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff23c5-45cc-4717-a004-59e6dc25f7d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:18:51.269987 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-6fff23c5-45cc-4717-a004-59e6dc25f7d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:18:51.353433 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fff23c5-45cc-4717-a004-59e6dc25f7d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:18:51.356062 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fff23c5-45cc-4717-a004-59e6dc25f7d6 service nova] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:18:51.367882 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1111/2226] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:18:51 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 16:18:53.312817 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ce11a9-de95-4595-888b-a4c5ad88a62f service neutron] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:18:53.315254 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-e8ce11a9-de95-4595-888b-a4c5ad88a62f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:18:53.383444 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ce11a9-de95-4595-888b-a4c5ad88a62f service neutron] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:18:53.391428 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4be376a-d147-4ef8-ac59-adab2fa56244 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:18:53.392580 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4be376a-d147-4ef8-ac59-adab2fa56244 service cinder] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:18:53.393364 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4be376a-d147-4ef8-ac59-adab2fa56244 service cinder] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:18:53.393927 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4be376a-d147-4ef8-ac59-adab2fa56244 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:18:53.397961 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-f4be376a-d147-4ef8-ac59-adab2fa56244 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:18:53.496829 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4be376a-d147-4ef8-ac59-adab2fa56244 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:18:53.499758 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4be376a-d147-4ef8-ac59-adab2fa56244 service cinder] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:18:53.517833 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1116/2227] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:18:53 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:19:01.363477 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=62 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:19:03.509001 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=60 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:19:07.088597 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fff23c5-45cc-4717-a004-59e6dc25f7d6 service nova] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:19:07.115178 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fff23c5-45cc-4717-a004-59e6dc25f7d6 service nova] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:19:07.122004 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064c4d06-7170-4bf2-98e6-b46a2c46941f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:19:07.123570 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064c4d06-7170-4bf2-98e6-b46a2c46941f service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:19:07.124012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064c4d06-7170-4bf2-98e6-b46a2c46941f service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:19:07.124799 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064c4d06-7170-4bf2-98e6-b46a2c46941f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:19:07.141709 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064c4d06-7170-4bf2-98e6-b46a2c46941f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:19:07.143181 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064c4d06-7170-4bf2-98e6-b46a2c46941f service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:19:07.168877 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1112/2228] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:19:07 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:19:09.707306 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42862db-4fbb-4874-9a18-33c6f0973e42 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:19:09.708330 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42862db-4fbb-4874-9a18-33c6f0973e42 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:19:09.708798 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42862db-4fbb-4874-9a18-33c6f0973e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:19:09.732962 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064c4d06-7170-4bf2-98e6-b46a2c46941f service neutron] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:19:09.737302 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-d42862db-4fbb-4874-9a18-33c6f0973e42 None None] MFA Rules not processed for user `8cd9c727c0b444cb83497ea4f78e24d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:19:09.739106 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-d42862db-4fbb-4874-9a18-33c6f0973e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:19:09.747227 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1117/2229] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:19:09 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:19:09.754429 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064c4d06-7170-4bf2-98e6-b46a2c46941f service neutron] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:19:09.767040 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43191cf-9e38-4a61-a21a-7ca002e6e3a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:19:09.767753 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43191cf-9e38-4a61-a21a-7ca002e6e3a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:19:09.768005 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43191cf-9e38-4a61-a21a-7ca002e6e3a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:19:09.768384 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43191cf-9e38-4a61-a21a-7ca002e6e3a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:19:09.772605 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-c43191cf-9e38-4a61-a21a-7ca002e6e3a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:19:09.812525 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43191cf-9e38-4a61-a21a-7ca002e6e3a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:19:09.816422 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43191cf-9e38-4a61-a21a-7ca002e6e3a8 service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:19:09.841474 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1113/2230] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:19:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:19:13.369218 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42862db-4fbb-4874-9a18-33c6f0973e42 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:19:13.415183 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42862db-4fbb-4874-9a18-33c6f0973e42 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:19:13.449214 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790a14f4-482d-405e-8000-06bc54530d53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:19:13.452656 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a14f4-482d-405e-8000-06bc54530d53 service nova] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:19:13.454548 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a14f4-482d-405e-8000-06bc54530d53 service nova] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:19:13.455695 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a14f4-482d-405e-8000-06bc54530d53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:19:13.460970 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-790a14f4-482d-405e-8000-06bc54530d53 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:19:13.504183 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790a14f4-482d-405e-8000-06bc54530d53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:19:13.506482 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790a14f4-482d-405e-8000-06bc54530d53 service nova] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:19:13.536864 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1118/2231] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:19:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 16:19:18.571214 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43191cf-9e38-4a61-a21a-7ca002e6e3a8 service neutron] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:19:18.602984 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43191cf-9e38-4a61-a21a-7ca002e6e3a8 service neutron] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:19:18.616862 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e401b16-3ab0-4e08-aaf1-a389b0b27fd8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:19:18.619599 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e401b16-3ab0-4e08-aaf1-a389b0b27fd8 service nova] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:19:18.621664 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e401b16-3ab0-4e08-aaf1-a389b0b27fd8 service nova] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:19:18.622508 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e401b16-3ab0-4e08-aaf1-a389b0b27fd8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:19:18.628256 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-2e401b16-3ab0-4e08-aaf1-a389b0b27fd8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:19:18.661909 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e401b16-3ab0-4e08-aaf1-a389b0b27fd8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:19:18.666015 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e401b16-3ab0-4e08-aaf1-a389b0b27fd8 service nova] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:19:18.689646 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1114/2232] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:19:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:19:19.828426 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd86cc8-7574-4dd2-8128-9e9167398785 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:19:19.829952 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd86cc8-7574-4dd2-8128-9e9167398785 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:19:19.830625 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd86cc8-7574-4dd2-8128-9e9167398785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:19:19.871080 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-ecd86cc8-7574-4dd2-8128-9e9167398785 None None] MFA Rules not processed for user `754c1b5cb47746cf93c52d6dc0b8fa01`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:19:19.874603 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-ecd86cc8-7574-4dd2-8128-9e9167398785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:19:19.889107 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1119/2233] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:19:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:19:20.883289 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e401b16-3ab0-4e08-aaf1-a389b0b27fd8 service nova] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:19:20.894256 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e401b16-3ab0-4e08-aaf1-a389b0b27fd8 service nova] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:19:20.898485 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667a436d-eb08-43e4-a7d0-389d3754e278 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:19:20.899197 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667a436d-eb08-43e4-a7d0-389d3754e278 service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:19:20.899543 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667a436d-eb08-43e4-a7d0-389d3754e278 service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:19:20.899820 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667a436d-eb08-43e4-a7d0-389d3754e278 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:19:20.901658 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-667a436d-eb08-43e4-a7d0-389d3754e278 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:19:20.915050 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667a436d-eb08-43e4-a7d0-389d3754e278 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:19:20.917766 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667a436d-eb08-43e4-a7d0-389d3754e278 service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:19:20.935225 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1115/2234] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:19:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:19:21.420982 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd86cc8-7574-4dd2-8128-9e9167398785 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:19:21.423358 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-ecd86cc8-7574-4dd2-8128-9e9167398785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:19:21.551040 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd86cc8-7574-4dd2-8128-9e9167398785 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:19:21.560434 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eec0d1-012e-42e3-ba17-e04c3f17cd2e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1a985495ed48a4acc48e2e6fc7f977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:19:21.561714 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eec0d1-012e-42e3-ba17-e04c3f17cd2e service placement] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:19:21.562185 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eec0d1-012e-42e3-ba17-e04c3f17cd2e service placement] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:19:21.562691 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eec0d1-012e-42e3-ba17-e04c3f17cd2e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:19:21.565762 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-c6eec0d1-012e-42e3-ba17-e04c3f17cd2e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:19:21.578550 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eec0d1-012e-42e3-ba17-e04c3f17cd2e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:19:21.580082 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eec0d1-012e-42e3-ba17-e04c3f17cd2e service placement] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:19:21.592149 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1120/2235] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:19:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:19:30.928214 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=24 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:19:31.342834 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667a436d-eb08-43e4-a7d0-389d3754e278 service neutron] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:19:31.365953 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667a436d-eb08-43e4-a7d0-389d3754e278 service neutron] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:19:31.376649 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a2d5d7-31f4-4b83-99f3-186d4959d702 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:19:31.378055 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2d5d7-31f4-4b83-99f3-186d4959d702 service nova] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:19:31.379132 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2d5d7-31f4-4b83-99f3-186d4959d702 service nova] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:19:31.379868 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2d5d7-31f4-4b83-99f3-186d4959d702 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:19:31.383320 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-79a2d5d7-31f4-4b83-99f3-186d4959d702 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:19:31.408906 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a2d5d7-31f4-4b83-99f3-186d4959d702 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:19:31.412714 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a2d5d7-31f4-4b83-99f3-186d4959d702 service nova] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:19:31.435415 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1116/2236] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:19:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:19:31.586630 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=44 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:19:41.425567 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=6 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:20:33.425673 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eec0d1-012e-42e3-ba17-e04c3f17cd2e service placement] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:20:33.436032 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eec0d1-012e-42e3-ba17-e04c3f17cd2e service placement] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:20:33.440020 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbb747f-e5e7-4a10-828e-8f38a71b31c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:20:33.440656 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb747f-e5e7-4a10-828e-8f38a71b31c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:20:33.440952 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb747f-e5e7-4a10-828e-8f38a71b31c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:20:33.441217 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb747f-e5e7-4a10-828e-8f38a71b31c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:20:33.443416 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-4bbb747f-e5e7-4a10-828e-8f38a71b31c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:20:33.517389 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbb747f-e5e7-4a10-828e-8f38a71b31c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:20:33.520250 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbb747f-e5e7-4a10-828e-8f38a71b31c5 service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:20:33.538237 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1121/2237] 10.222.0.57 () {60 vars in 1396 bytes} [Mon Sep 19 16:20:33 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3254 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:20:43.530130 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=28 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:21:07.980372 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c9d2e-fe7e-457f-820f-f2db142a3e3e None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:21:07.981034 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c9d2e-fe7e-457f-820f-f2db142a3e3e None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:21:07.981178 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c9d2e-fe7e-457f-820f-f2db142a3e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:21:08.003705 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-074c9d2e-fe7e-457f-820f-f2db142a3e3e None None] MFA Rules not processed for user `fd36089b3c2141218660ab2458376798`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:21:08.005047 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-074c9d2e-fe7e-457f-820f-f2db142a3e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:21:08.015606 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1117/2238] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:21:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:21:17.997823 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=3 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:22:35.790114 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbb747f-e5e7-4a10-828e-8f38a71b31c5 service neutron] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:22:35.861051 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbb747f-e5e7-4a10-828e-8f38a71b31c5 service neutron] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:22:35.866995 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98deb532-ba6e-4ba3-a56f-2e6032ad60cd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1a985495ed48a4acc48e2e6fc7f977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:22:35.867614 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98deb532-ba6e-4ba3-a56f-2e6032ad60cd service placement] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:22:35.867885 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98deb532-ba6e-4ba3-a56f-2e6032ad60cd service placement] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:22:35.868166 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98deb532-ba6e-4ba3-a56f-2e6032ad60cd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:22:35.870078 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-98deb532-ba6e-4ba3-a56f-2e6032ad60cd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:22:35.912315 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98deb532-ba6e-4ba3-a56f-2e6032ad60cd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:22:35.914005 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98deb532-ba6e-4ba3-a56f-2e6032ad60cd service placement] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:22:35.925645 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1122/2239] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:22:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:22:45.920509 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=36 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:23:07.618856 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074c9d2e-fe7e-457f-820f-f2db142a3e3e None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:23:07.652485 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074c9d2e-fe7e-457f-820f-f2db142a3e3e None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:23:07.656715 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8bef67-7c4d-4e64-9904-7e1ee51579d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:23:07.657374 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8bef67-7c4d-4e64-9904-7e1ee51579d4 service cinder] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:23:07.657661 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8bef67-7c4d-4e64-9904-7e1ee51579d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:23:07.657994 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8bef67-7c4d-4e64-9904-7e1ee51579d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:23:07.749634 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8bef67-7c4d-4e64-9904-7e1ee51579d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:23:07.751122 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8bef67-7c4d-4e64-9904-7e1ee51579d4 service cinder] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:23:07.762536 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1118/2240] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:23:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:23:07.792304 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98deb532-ba6e-4ba3-a56f-2e6032ad60cd service placement] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:23:07.845005 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98deb532-ba6e-4ba3-a56f-2e6032ad60cd service placement] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:23:07.854918 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fc908f-8d7e-41de-8cd2-8f8954705773 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:23:07.855889 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc908f-8d7e-41de-8cd2-8f8954705773 service nova] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:23:07.856343 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc908f-8d7e-41de-8cd2-8f8954705773 service nova] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:23:07.856798 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc908f-8d7e-41de-8cd2-8f8954705773 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:23:07.870301 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fc908f-8d7e-41de-8cd2-8f8954705773 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:23:07.873182 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fc908f-8d7e-41de-8cd2-8f8954705773 service nova] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:23:07.890154 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1123/2241] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:23:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:23:17.757931 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=34 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:23:17.881971 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=13 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:23:47.818259 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8bef67-7c4d-4e64-9904-7e1ee51579d4 service cinder] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:23:47.840376 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8bef67-7c4d-4e64-9904-7e1ee51579d4 service cinder] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:23:47.844156 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542e29fe-a364-4d92-96d5-d68be7db223f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:23:47.844766 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e29fe-a364-4d92-96d5-d68be7db223f service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:23:47.845047 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e29fe-a364-4d92-96d5-d68be7db223f service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:23:47.845321 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e29fe-a364-4d92-96d5-d68be7db223f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:23:47.858315 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542e29fe-a364-4d92-96d5-d68be7db223f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:23:47.860177 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542e29fe-a364-4d92-96d5-d68be7db223f service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:23:47.873992 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1119/2242] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:23:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:23:51.315343 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fc908f-8d7e-41de-8cd2-8f8954705773 service nova] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:23:51.335446 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fc908f-8d7e-41de-8cd2-8f8954705773 service nova] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:23:51.341439 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1335bb9a-16fd-498c-af43-e0e85effb520 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:23:51.342120 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335bb9a-16fd-498c-af43-e0e85effb520 service nova] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:23:51.342468 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335bb9a-16fd-498c-af43-e0e85effb520 service nova] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:23:51.342789 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335bb9a-16fd-498c-af43-e0e85effb520 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:23:51.352616 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1335bb9a-16fd-498c-af43-e0e85effb520 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:23:51.354519 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1335bb9a-16fd-498c-af43-e0e85effb520 service nova] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:23:51.388641 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1124/2243] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:23:51 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 16:23:57.868353 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=10 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:24:01.385398 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=13 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:24:07.673138 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542e29fe-a364-4d92-96d5-d68be7db223f service neutron] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:24:07.692632 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542e29fe-a364-4d92-96d5-d68be7db223f service neutron] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:24:07.702149 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353c459d-0cf2-4cfd-8db1-d41ceb990a79 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:24:07.703119 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353c459d-0cf2-4cfd-8db1-d41ceb990a79 service nova] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:24:07.703559 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353c459d-0cf2-4cfd-8db1-d41ceb990a79 service nova] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:24:07.703988 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353c459d-0cf2-4cfd-8db1-d41ceb990a79 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:24:07.717596 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353c459d-0cf2-4cfd-8db1-d41ceb990a79 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:24:07.720322 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353c459d-0cf2-4cfd-8db1-d41ceb990a79 service nova] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:24:07.772898 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1120/2244] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:24:07 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:24:17.767640 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=13 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:24:44.500076 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1335bb9a-16fd-498c-af43-e0e85effb520 service nova] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:24:44.510921 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1335bb9a-16fd-498c-af43-e0e85effb520 service nova] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:24:44.514827 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43e552a-fad2-4e62-92da-35f651303be6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:24:44.515461 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e552a-fad2-4e62-92da-35f651303be6 service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:24:44.515744 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e552a-fad2-4e62-92da-35f651303be6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:24:44.516071 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e552a-fad2-4e62-92da-35f651303be6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:24:44.524877 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43e552a-fad2-4e62-92da-35f651303be6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:24:44.526826 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43e552a-fad2-4e62-92da-35f651303be6 service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:24:44.537985 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1125/2245] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:24:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:24:54.532365 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=6 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:25:08.369405 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353c459d-0cf2-4cfd-8db1-d41ceb990a79 service nova] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:25:08.384118 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353c459d-0cf2-4cfd-8db1-d41ceb990a79 service nova] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:25:08.389135 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daf4b57-d181-4733-a19c-f55b4d8cecae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:25:08.390051 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daf4b57-d181-4733-a19c-f55b4d8cecae service cinder] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:25:08.390349 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daf4b57-d181-4733-a19c-f55b4d8cecae service cinder] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:25:08.390649 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daf4b57-d181-4733-a19c-f55b4d8cecae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:25:08.402599 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daf4b57-d181-4733-a19c-f55b4d8cecae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:25:08.404038 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daf4b57-d181-4733-a19c-f55b4d8cecae service cinder] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:25:08.416261 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1121/2246] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:25:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:25:08.462654 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43e552a-fad2-4e62-92da-35f651303be6 service neutron] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:25:08.472114 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43e552a-fad2-4e62-92da-35f651303be6 service neutron] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:25:08.475940 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa1e1f3-b195-4ac1-8e2f-3e868a6165fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:25:08.476550 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa1e1f3-b195-4ac1-8e2f-3e868a6165fb service cinder] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:25:08.476831 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa1e1f3-b195-4ac1-8e2f-3e868a6165fb service cinder] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:25:08.477102 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa1e1f3-b195-4ac1-8e2f-3e868a6165fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:25:08.486504 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa1e1f3-b195-4ac1-8e2f-3e868a6165fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:25:08.487958 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa1e1f3-b195-4ac1-8e2f-3e868a6165fb service cinder] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:25:08.524134 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1126/2247] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:25:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:25:08.592948 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daf4b57-d181-4733-a19c-f55b4d8cecae service cinder] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:25:08.601917 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daf4b57-d181-4733-a19c-f55b4d8cecae service cinder] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:25:08.606030 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffc8cf9-abae-4eb8-8ea4-c125e32e2ffb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:25:08.606679 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc8cf9-abae-4eb8-8ea4-c125e32e2ffb service cinder] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:25:08.606957 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc8cf9-abae-4eb8-8ea4-c125e32e2ffb service cinder] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:25:08.607221 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc8cf9-abae-4eb8-8ea4-c125e32e2ffb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:25:08.618616 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffc8cf9-abae-4eb8-8ea4-c125e32e2ffb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:25:08.620880 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffc8cf9-abae-4eb8-8ea4-c125e32e2ffb service cinder] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:25:08.655949 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1122/2248] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:25:08 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 16:25:18.521347 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=13 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:25:18.653560 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=19 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:27:08.937385 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa1e1f3-b195-4ac1-8e2f-3e868a6165fb service cinder] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:27:08.952893 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa1e1f3-b195-4ac1-8e2f-3e868a6165fb service cinder] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:27:08.958683 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fac3a22-5186-4618-98b5-d04ad0464cc0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:27:08.959299 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac3a22-5186-4618-98b5-d04ad0464cc0 service nova] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:27:08.959582 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac3a22-5186-4618-98b5-d04ad0464cc0 service nova] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:27:08.959859 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac3a22-5186-4618-98b5-d04ad0464cc0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:27:08.968919 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fac3a22-5186-4618-98b5-d04ad0464cc0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:27:08.970659 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fac3a22-5186-4618-98b5-d04ad0464cc0 service nova] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:27:09.005477 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1127/2249] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:27:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:27:19.004149 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=13 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:27:40.783403 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffc8cf9-abae-4eb8-8ea4-c125e32e2ffb service cinder] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:27:40.803696 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffc8cf9-abae-4eb8-8ea4-c125e32e2ffb service cinder] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:27:40.818961 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022e025c-0393-4d06-893b-9d3cc6e1e9cb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1a985495ed48a4acc48e2e6fc7f977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:27:40.819497 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e025c-0393-4d06-893b-9d3cc6e1e9cb service placement] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:27:40.819801 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e025c-0393-4d06-893b-9d3cc6e1e9cb service placement] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:27:40.820138 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e025c-0393-4d06-893b-9d3cc6e1e9cb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:27:40.829484 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022e025c-0393-4d06-893b-9d3cc6e1e9cb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:27:40.831182 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022e025c-0393-4d06-893b-9d3cc6e1e9cb service placement] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:27:40.841330 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1123/2250] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:27:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:27:50.836929 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=6 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:29:09.214403 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fac3a22-5186-4618-98b5-d04ad0464cc0 service nova] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:29:09.215953 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-2fac3a22-5186-4618-98b5-d04ad0464cc0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:29:09.285463 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fac3a22-5186-4618-98b5-d04ad0464cc0 service nova] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:29:09.289072 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a2f1ba-0060-4106-a66b-8186de7b7dd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:29:09.289670 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a2f1ba-0060-4106-a66b-8186de7b7dd5 service cinder] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:29:09.289954 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a2f1ba-0060-4106-a66b-8186de7b7dd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:29:09.290226 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a2f1ba-0060-4106-a66b-8186de7b7dd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:29:09.292144 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-52a2f1ba-0060-4106-a66b-8186de7b7dd5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:29:09.373560 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a2f1ba-0060-4106-a66b-8186de7b7dd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:29:09.375119 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a2f1ba-0060-4106-a66b-8186de7b7dd5 service cinder] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:29:09.385874 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1128/2251] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:29:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:29:09.393347 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022e025c-0393-4d06-893b-9d3cc6e1e9cb service placement] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:29:09.402345 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022e025c-0393-4d06-893b-9d3cc6e1e9cb service placement] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:29:09.406630 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34871e4-bf94-42b5-8a8a-325ec7dbe27a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:29:09.407171 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34871e4-bf94-42b5-8a8a-325ec7dbe27a service cinder] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:29:09.407448 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34871e4-bf94-42b5-8a8a-325ec7dbe27a service cinder] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:29:09.407717 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34871e4-bf94-42b5-8a8a-325ec7dbe27a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:29:09.409704 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-f34871e4-bf94-42b5-8a8a-325ec7dbe27a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:29:09.443670 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34871e4-bf94-42b5-8a8a-325ec7dbe27a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:29:09.445085 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34871e4-bf94-42b5-8a8a-325ec7dbe27a service cinder] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:29:09.464275 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1124/2252] 10.222.0.57 () {60 vars in 1396 bytes} [Mon Sep 19 16:29:09 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3260 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:29:09.515713 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a2f1ba-0060-4106-a66b-8186de7b7dd5 service cinder] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:29:09.524624 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a2f1ba-0060-4106-a66b-8186de7b7dd5 service cinder] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:29:09.528441 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d003c3e9-2606-41f9-a9a9-5fc647374328 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:29:09.529211 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d003c3e9-2606-41f9-a9a9-5fc647374328 service cinder] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:29:09.529496 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d003c3e9-2606-41f9-a9a9-5fc647374328 service cinder] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:29:09.529772 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d003c3e9-2606-41f9-a9a9-5fc647374328 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:29:09.532228 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-d003c3e9-2606-41f9-a9a9-5fc647374328 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:29:09.607084 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d003c3e9-2606-41f9-a9a9-5fc647374328 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:29:09.608691 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d003c3e9-2606-41f9-a9a9-5fc647374328 service cinder] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:29:09.618339 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1129/2253] 10.222.0.57 () {60 vars in 1396 bytes} [Mon Sep 19 16:29:09 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3268 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:29:09.710047 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34871e4-bf94-42b5-8a8a-325ec7dbe27a service cinder] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:29:09.718465 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34871e4-bf94-42b5-8a8a-325ec7dbe27a service cinder] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:29:09.721999 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499112f3-184f-49d7-a8fe-d33b688a2778 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:29:09.722636 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499112f3-184f-49d7-a8fe-d33b688a2778 service cinder] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:29:09.722918 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499112f3-184f-49d7-a8fe-d33b688a2778 service cinder] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:29:09.723195 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499112f3-184f-49d7-a8fe-d33b688a2778 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:29:09.725022 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-499112f3-184f-49d7-a8fe-d33b688a2778 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:29:09.769995 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d003c3e9-2606-41f9-a9a9-5fc647374328 service cinder] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:29:09.771975 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-d003c3e9-2606-41f9-a9a9-5fc647374328 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:29:09.789850 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d003c3e9-2606-41f9-a9a9-5fc647374328 service cinder] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:29:09.795828 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f43863b-e336-4ae4-8f72-a3c7c07d6f15 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:29:09.796435 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f43863b-e336-4ae4-8f72-a3c7c07d6f15 service nova] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:29:09.796717 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f43863b-e336-4ae4-8f72-a3c7c07d6f15 service nova] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:29:09.796995 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f43863b-e336-4ae4-8f72-a3c7c07d6f15 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:29:09.799029 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-1f43863b-e336-4ae4-8f72-a3c7c07d6f15 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:29:09.801854 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:29:09.812952 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499112f3-184f-49d7-a8fe-d33b688a2778 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:29:09.814497 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499112f3-184f-49d7-a8fe-d33b688a2778 service cinder] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:29:09.823779 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1125/2254] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:29:09 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:29:09.890976 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f43863b-e336-4ae4-8f72-a3c7c07d6f15 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:29:09.892532 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f43863b-e336-4ae4-8f72-a3c7c07d6f15 service nova] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:29:09.923732 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1130/2255] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:29:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 16:29:18.132583 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499112f3-184f-49d7-a8fe-d33b688a2778 service cinder] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:29:18.147958 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499112f3-184f-49d7-a8fe-d33b688a2778 service cinder] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:29:18.154397 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48da394-c304-47e4-af0c-0358632a0214 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:29:18.155486 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48da394-c304-47e4-af0c-0358632a0214 service nova] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:29:18.155988 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48da394-c304-47e4-af0c-0358632a0214 service nova] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:29:18.156352 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48da394-c304-47e4-af0c-0358632a0214 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:29:18.158635 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-c48da394-c304-47e4-af0c-0358632a0214 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:29:18.174439 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48da394-c304-47e4-af0c-0358632a0214 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:29:18.176245 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48da394-c304-47e4-af0c-0358632a0214 service nova] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:29:18.187842 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1126/2256] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:29:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:29:19.909083 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=38 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:29:28.182837 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=58 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:29:48.464350 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f43863b-e336-4ae4-8f72-a3c7c07d6f15 service nova] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:29:48.466194 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-1f43863b-e336-4ae4-8f72-a3c7c07d6f15 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:29:48.542655 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f43863b-e336-4ae4-8f72-a3c7c07d6f15 service nova] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:29:48.547097 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606256de-d8e4-45c1-8726-210c7963688e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:29:48.547719 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606256de-d8e4-45c1-8726-210c7963688e service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:29:48.547999 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606256de-d8e4-45c1-8726-210c7963688e service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:29:48.548277 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606256de-d8e4-45c1-8726-210c7963688e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:29:48.554091 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-606256de-d8e4-45c1-8726-210c7963688e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:29:48.566166 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606256de-d8e4-45c1-8726-210c7963688e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:29:48.567696 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606256de-d8e4-45c1-8726-210c7963688e service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:29:48.577122 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1131/2257] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:29:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:29:58.572926 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=31 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:31:10.171932 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48da394-c304-47e4-af0c-0358632a0214 service nova] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:31:10.182569 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48da394-c304-47e4-af0c-0358632a0214 service nova] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:31:10.186917 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0358584-f649-4a79-ac6c-8e7ac805397d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:31:10.187501 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0358584-f649-4a79-ac6c-8e7ac805397d service cinder] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:31:10.187777 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0358584-f649-4a79-ac6c-8e7ac805397d service cinder] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:31:10.188048 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0358584-f649-4a79-ac6c-8e7ac805397d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:31:10.189876 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-d0358584-f649-4a79-ac6c-8e7ac805397d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:31:10.270765 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0358584-f649-4a79-ac6c-8e7ac805397d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:31:10.272212 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0358584-f649-4a79-ac6c-8e7ac805397d service cinder] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:31:10.282085 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1127/2258] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:31:10 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:31:10.413438 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606256de-d8e4-45c1-8726-210c7963688e service neutron] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:31:10.423845 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606256de-d8e4-45c1-8726-210c7963688e service neutron] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:31:10.427742 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e412939-af81-4359-9453-351f33e8b261 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:31:10.428345 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e412939-af81-4359-9453-351f33e8b261 service cinder] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:31:10.428620 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e412939-af81-4359-9453-351f33e8b261 service cinder] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:31:10.428892 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e412939-af81-4359-9453-351f33e8b261 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:31:10.430784 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-5e412939-af81-4359-9453-351f33e8b261 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:31:10.503057 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e412939-af81-4359-9453-351f33e8b261 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:31:10.504495 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e412939-af81-4359-9453-351f33e8b261 service cinder] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:31:10.514987 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1132/2259] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:31:10 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:31:10.558391 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0358584-f649-4a79-ac6c-8e7ac805397d service cinder] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:31:10.567006 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0358584-f649-4a79-ac6c-8e7ac805397d service cinder] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:31:10.571566 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a085cdfd-548d-4b9f-a733-43f664006b09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:31:10.572213 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a085cdfd-548d-4b9f-a733-43f664006b09 service cinder] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:31:10.572489 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a085cdfd-548d-4b9f-a733-43f664006b09 service cinder] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:31:10.572766 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a085cdfd-548d-4b9f-a733-43f664006b09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:31:10.574976 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-a085cdfd-548d-4b9f-a733-43f664006b09 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:31:10.589875 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a085cdfd-548d-4b9f-a733-43f664006b09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:31:10.591862 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a085cdfd-548d-4b9f-a733-43f664006b09 service cinder] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:31:10.603342 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1128/2260] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:31:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:31:17.826470 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e412939-af81-4359-9453-351f33e8b261 service cinder] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:31:17.828296 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-5e412939-af81-4359-9453-351f33e8b261 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:31:17.842946 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e412939-af81-4359-9453-351f33e8b261 service cinder] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:31:17.846645 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58567337-bcaa-4798-aa81-281a3d031163 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:31:17.847248 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58567337-bcaa-4798-aa81-281a3d031163 service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:31:17.847536 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58567337-bcaa-4798-aa81-281a3d031163 service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:31:17.847809 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58567337-bcaa-4798-aa81-281a3d031163 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:31:17.849565 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-58567337-bcaa-4798-aa81-281a3d031163 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:31:17.943273 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58567337-bcaa-4798-aa81-281a3d031163 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:31:17.944707 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58567337-bcaa-4798-aa81-281a3d031163 service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:31:17.964491 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1133/2261] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:31:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:31:20.598308 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=39 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:31:21.287385 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a085cdfd-548d-4b9f-a733-43f664006b09 service cinder] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:31:21.288818 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-a085cdfd-548d-4b9f-a733-43f664006b09 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:31:21.360789 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58567337-bcaa-4798-aa81-281a3d031163 service neutron] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:31:21.362284 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-58567337-bcaa-4798-aa81-281a3d031163 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:31:21.374526 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58567337-bcaa-4798-aa81-281a3d031163 service neutron] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:31:21.379140 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e45f3d4-33b8-4cb9-b3da-c3a1dca2356b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:31:21.379768 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e45f3d4-33b8-4cb9-b3da-c3a1dca2356b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:31:21.380045 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e45f3d4-33b8-4cb9-b3da-c3a1dca2356b admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:31:21.380314 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e45f3d4-33b8-4cb9-b3da-c3a1dca2356b admin admin] PATH_INFO: `/v3/users/fd36089b3c2141218660ab2458376798` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:31:21.381440 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e45f3d4-33b8-4cb9-b3da-c3a1dca2356b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd36089b3c2141218660ab2458376798)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:31:21.383362 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e45f3d4-33b8-4cb9-b3da-c3a1dca2356b admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:31:21.384110 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a085cdfd-548d-4b9f-a733-43f664006b09 service cinder] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:31:21.390719 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee81a2ca-4415-4cb4-ac51-524774270b80 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1a985495ed48a4acc48e2e6fc7f977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:31:21.391325 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee81a2ca-4415-4cb4-ac51-524774270b80 service placement] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:31:21.391607 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee81a2ca-4415-4cb4-ac51-524774270b80 service placement] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:31:21.391883 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee81a2ca-4415-4cb4-ac51-524774270b80 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:31:21.393607 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-ee81a2ca-4415-4cb4-ac51-524774270b80 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:31:21.406004 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee81a2ca-4415-4cb4-ac51-524774270b80 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:31:21.407661 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee81a2ca-4415-4cb4-ac51-524774270b80 service placement] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:31:21.418611 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1129/2262] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:31:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:31:21.443962 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-6e45f3d4-33b8-4cb9-b3da-c3a1dca2356b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd36089b3c2141218660ab2458376798'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:31:21.444631 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-6e45f3d4-33b8-4cb9-b3da-c3a1dca2356b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd36089b3c2141218660ab2458376798'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:31:21.445816 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:31:21.455498 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-6e45f3d4-33b8-4cb9-b3da-c3a1dca2356b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd36089b3c2141218660ab2458376798'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:31:21.467748 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-6e45f3d4-33b8-4cb9-b3da-c3a1dca2356b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd36089b3c2141218660ab2458376798'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:31:21.468446 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-6e45f3d4-33b8-4cb9-b3da-c3a1dca2356b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd36089b3c2141218660ab2458376798'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:31:21.481166 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1134/2263] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:31:21 2022] DELETE /identity/v3/users/fd36089b3c2141218660ab2458376798 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:31:21.655352 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee81a2ca-4415-4cb4-ac51-524774270b80 service placement] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:31:21.666441 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee81a2ca-4415-4cb4-ac51-524774270b80 service placement] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:31:21.741955 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dde208-8809-4653-8e4f-214649797462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:31:21.742612 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dde208-8809-4653-8e4f-214649797462 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:31:21.742890 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dde208-8809-4653-8e4f-214649797462 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:31:21.743161 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dde208-8809-4653-8e4f-214649797462 admin admin] PATH_INFO: `/v3/projects/a1842253d6634f1f8b93bfa5be2ddd5f` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:31:21.744277 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dde208-8809-4653-8e4f-214649797462 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1842253d6634f1f8b93bfa5be2ddd5f)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:31:21.746154 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dde208-8809-4653-8e4f-214649797462 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:31:21.794638 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-43dde208-8809-4653-8e4f-214649797462 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'request_id': 'req-43dde208-8809-4653-8e4f-214649797462'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:31:21.800876 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-43dde208-8809-4653-8e4f-214649797462 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'request_id': 'req-43dde208-8809-4653-8e4f-214649797462'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:31:21.806352 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-43dde208-8809-4653-8e4f-214649797462 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'request_id': 'req-43dde208-8809-4653-8e4f-214649797462'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:31:21.819341 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-43dde208-8809-4653-8e4f-214649797462 admin admin] The token cache is being invalidate because project a1842253d6634f1f8b93bfa5be2ddd5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81618) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:31:21.819632 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-43dde208-8809-4653-8e4f-214649797462 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:31:21.820257 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-43dde208-8809-4653-8e4f-214649797462 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:31:21.821814 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1130/2264] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:31:21 2022] DELETE /identity/v3/projects/a1842253d6634f1f8b93bfa5be2ddd5f => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:31:21.828162 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d36da77-8fdf-4648-b4d1-33c2016cf66d None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:31:21.828535 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d36da77-8fdf-4648-b4d1-33c2016cf66d None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:31:21.828819 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d36da77-8fdf-4648-b4d1-33c2016cf66d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:31:21.886765 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-3d36da77-8fdf-4648-b4d1-33c2016cf66d None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:31:21.887992 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-3d36da77-8fdf-4648-b4d1-33c2016cf66d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:31:21.946874 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1135/2265] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:31:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:31:21.954864 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a9efca-986b-42ce-a818-7d3e933cb003 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:31:21.955162 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a9efca-986b-42ce-a818-7d3e933cb003 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:31:21.955440 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a9efca-986b-42ce-a818-7d3e933cb003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:31:21.975015 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-39a9efca-986b-42ce-a818-7d3e933cb003 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:31:21.976227 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-39a9efca-986b-42ce-a818-7d3e933cb003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:31:21.984648 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1131/2266] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:31:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:31:21.988984 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d36da77-8fdf-4648-b4d1-33c2016cf66d None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:31:21.999828 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d36da77-8fdf-4648-b4d1-33c2016cf66d None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:31:22.004426 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94c13c8-bc21-4391-a40c-1cae5f0355a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:31:22.005026 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c13c8-bc21-4391-a40c-1cae5f0355a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:31:22.005336 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c13c8-bc21-4391-a40c-1cae5f0355a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:31:22.005612 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c13c8-bc21-4391-a40c-1cae5f0355a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:31:22.006265 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94c13c8-bc21-4391-a40c-1cae5f0355a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:31:22.007204 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94c13c8-bc21-4391-a40c-1cae5f0355a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:31:22.008811 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94c13c8-bc21-4391-a40c-1cae5f0355a2 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:31:22.021043 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1136/2267] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:31:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:31:22.025431 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a9efca-986b-42ce-a818-7d3e933cb003 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:31:22.034849 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a9efca-986b-42ce-a818-7d3e933cb003 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:31:22.039724 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3afa75-f566-4e97-8b17-933acceea268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:31:22.040318 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3afa75-f566-4e97-8b17-933acceea268 admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:31:22.040598 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3afa75-f566-4e97-8b17-933acceea268 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:31:22.040891 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3afa75-f566-4e97-8b17-933acceea268 admin admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:31:22.041558 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3afa75-f566-4e97-8b17-933acceea268 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:31:22.043501 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3afa75-f566-4e97-8b17-933acceea268 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:31:22.057399 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:31:22.091443 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1132/2268] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:31:22 2022] POST /identity/v3/projects => generated 365 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:31:22.095857 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94c13c8-bc21-4391-a40c-1cae5f0355a2 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:31:22.106090 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94c13c8-bc21-4391-a40c-1cae5f0355a2 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:31:22.161251 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c91bd4a-c902-4734-8d2a-57faae62b556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:31:22.161854 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c91bd4a-c902-4734-8d2a-57faae62b556 admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:31:22.162118 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c91bd4a-c902-4734-8d2a-57faae62b556 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:31:22.162454 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c91bd4a-c902-4734-8d2a-57faae62b556 admin admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:31:22.163240 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c91bd4a-c902-4734-8d2a-57faae62b556 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:31:22.164990 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c91bd4a-c902-4734-8d2a-57faae62b556 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:31:22.189928 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1137/2269] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:31:22 2022] POST /identity/v3/users => generated 283 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:31:22.194576 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3afa75-f566-4e97-8b17-933acceea268 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:31:22.208540 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3afa75-f566-4e97-8b17-933acceea268 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:31:22.214516 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9198fbba-36a2-466c-919b-9c1972ad007e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:31:22.215351 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9198fbba-36a2-466c-919b-9c1972ad007e admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:31:22.215691 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9198fbba-36a2-466c-919b-9c1972ad007e admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:31:22.215944 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9198fbba-36a2-466c-919b-9c1972ad007e admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:31:22.216649 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9198fbba-36a2-466c-919b-9c1972ad007e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:31:22.218325 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9198fbba-36a2-466c-919b-9c1972ad007e admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:31:22.229536 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1133/2270] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:31:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:31:22.234215 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c91bd4a-c902-4734-8d2a-57faae62b556 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:31:22.244322 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c91bd4a-c902-4734-8d2a-57faae62b556 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:31:22.249627 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14edfe87-c54e-4e09-ac78-97409dd4cf32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:31:22.250219 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14edfe87-c54e-4e09-ac78-97409dd4cf32 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:31:22.250532 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14edfe87-c54e-4e09-ac78-97409dd4cf32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:31:22.250807 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14edfe87-c54e-4e09-ac78-97409dd4cf32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:31:22.251440 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14edfe87-c54e-4e09-ac78-97409dd4cf32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:31:22.253013 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14edfe87-c54e-4e09-ac78-97409dd4cf32 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:31:22.258593 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:31:22.264844 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1138/2271] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:31:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:31:22.268770 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9198fbba-36a2-466c-919b-9c1972ad007e admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:31:22.278574 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9198fbba-36a2-466c-919b-9c1972ad007e admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:31:22.283174 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22919dab-8d28-4228-a1e6-3ba4d8719ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:31:22.283819 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22919dab-8d28-4228-a1e6-3ba4d8719ad8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:31:22.284092 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22919dab-8d28-4228-a1e6-3ba4d8719ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:31:22.284363 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22919dab-8d28-4228-a1e6-3ba4d8719ad8 admin admin] PATH_INFO: `/v3/projects/4eb71767817a4baa994f8870ff2b0fa2/users/bb79fb8faeb2450aa9968aed39fee04b/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:31:22.299691 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22919dab-8d28-4228-a1e6-3ba4d8719ad8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4eb71767817a4baa994f8870ff2b0fa2, user_id=bb79fb8faeb2450aa9968aed39fee04b, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:31:22.302595 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22919dab-8d28-4228-a1e6-3ba4d8719ad8 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:31:22.316689 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1134/2272] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:31:22 2022] PUT /identity/v3/projects/4eb71767817a4baa994f8870ff2b0fa2/users/bb79fb8faeb2450aa9968aed39fee04b/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:31:22.328087 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622ab54-b990-45b2-b2ea-15b732e56b3f None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:31:22.328372 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622ab54-b990-45b2-b2ea-15b732e56b3f None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:31:22.328646 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622ab54-b990-45b2-b2ea-15b732e56b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:31:22.356708 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-8622ab54-b990-45b2-b2ea-15b732e56b3f None None] MFA Rules not processed for user `bb79fb8faeb2450aa9968aed39fee04b`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:31:22.358582 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-8622ab54-b990-45b2-b2ea-15b732e56b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:31:22.421754 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1139/2273] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:31:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:31:32.312844 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=22,keystone:INSERT=2 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:31:32.418185 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=32 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:32:22.487715 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22919dab-8d28-4228-a1e6-3ba4d8719ad8 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:32:22.564184 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22919dab-8d28-4228-a1e6-3ba4d8719ad8 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:32:22.568855 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c14761-9e26-4d59-985a-ce75cada75cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:32:22.569482 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c14761-9e26-4d59-985a-ce75cada75cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:32:22.569763 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c14761-9e26-4d59-985a-ce75cada75cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:32:22.570039 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c14761-9e26-4d59-985a-ce75cada75cd admin admin] PATH_INFO: `/v3/users/bb79fb8faeb2450aa9968aed39fee04b` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:32:22.571191 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c14761-9e26-4d59-985a-ce75cada75cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb79fb8faeb2450aa9968aed39fee04b)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:32:22.573100 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c14761-9e26-4d59-985a-ce75cada75cd admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:32:22.617570 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-40c14761-9e26-4d59-985a-ce75cada75cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb79fb8faeb2450aa9968aed39fee04b'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:32:22.624952 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-40c14761-9e26-4d59-985a-ce75cada75cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb79fb8faeb2450aa9968aed39fee04b'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:32:22.636512 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-40c14761-9e26-4d59-985a-ce75cada75cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb79fb8faeb2450aa9968aed39fee04b'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:32:22.642879 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-40c14761-9e26-4d59-985a-ce75cada75cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb79fb8faeb2450aa9968aed39fee04b'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:32:22.643708 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-40c14761-9e26-4d59-985a-ce75cada75cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb79fb8faeb2450aa9968aed39fee04b'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:32:22.649705 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1135/2274] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:32:22 2022] DELETE /identity/v3/users/bb79fb8faeb2450aa9968aed39fee04b => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:32:22.654719 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a179a0da-7d26-4bb0-ab3e-f60084de533c None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:32:22.655219 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a179a0da-7d26-4bb0-ab3e-f60084de533c None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:32:22.655326 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a179a0da-7d26-4bb0-ab3e-f60084de533c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:32:22.674614 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-a179a0da-7d26-4bb0-ab3e-f60084de533c None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:32:22.675838 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-a179a0da-7d26-4bb0-ab3e-f60084de533c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:32:22.738355 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1140/2275] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:32:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:32:22.748356 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c14761-9e26-4d59-985a-ce75cada75cd admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:32:22.749743 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-40c14761-9e26-4d59-985a-ce75cada75cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:32:22.808311 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c14761-9e26-4d59-985a-ce75cada75cd admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:32:22.812032 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581bfe7c-3de2-498c-9c35-52260828a4cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:32:22.812633 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581bfe7c-3de2-498c-9c35-52260828a4cb service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:32:22.812915 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581bfe7c-3de2-498c-9c35-52260828a4cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:32:22.813218 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581bfe7c-3de2-498c-9c35-52260828a4cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:32:22.822480 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581bfe7c-3de2-498c-9c35-52260828a4cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:32:22.823896 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581bfe7c-3de2-498c-9c35-52260828a4cb service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:32:22.833273 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1136/2276] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:32:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:32:23.104768 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a179a0da-7d26-4bb0-ab3e-f60084de533c None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:32:23.106281 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-a179a0da-7d26-4bb0-ab3e-f60084de533c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:32:23.118303 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a179a0da-7d26-4bb0-ab3e-f60084de533c None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:32:23.123246 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa3c2d4-2384-4271-8917-db1fd7336f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:32:23.123871 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa3c2d4-2384-4271-8917-db1fd7336f08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:32:23.124145 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa3c2d4-2384-4271-8917-db1fd7336f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:32:23.124416 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa3c2d4-2384-4271-8917-db1fd7336f08 admin admin] PATH_INFO: `/v3/projects/4eb71767817a4baa994f8870ff2b0fa2` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:32:23.125532 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa3c2d4-2384-4271-8917-db1fd7336f08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4eb71767817a4baa994f8870ff2b0fa2)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:32:23.127458 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa3c2d4-2384-4271-8917-db1fd7336f08 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:32:23.176900 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-8aa3c2d4-2384-4271-8917-db1fd7336f08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4eb71767817a4baa994f8870ff2b0fa2', 'request_id': 'req-8aa3c2d4-2384-4271-8917-db1fd7336f08'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:32:23.183037 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-8aa3c2d4-2384-4271-8917-db1fd7336f08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4eb71767817a4baa994f8870ff2b0fa2', 'request_id': 'req-8aa3c2d4-2384-4271-8917-db1fd7336f08'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:32:23.188649 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-8aa3c2d4-2384-4271-8917-db1fd7336f08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4eb71767817a4baa994f8870ff2b0fa2', 'request_id': 'req-8aa3c2d4-2384-4271-8917-db1fd7336f08'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:32:23.201411 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-8aa3c2d4-2384-4271-8917-db1fd7336f08 admin admin] The token cache is being invalidate because project 4eb71767817a4baa994f8870ff2b0fa2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81617) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:32:23.201700 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-8aa3c2d4-2384-4271-8917-db1fd7336f08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:32:23.202323 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-8aa3c2d4-2384-4271-8917-db1fd7336f08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:32:23.203836 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1141/2277] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:32:23 2022] DELETE /identity/v3/projects/4eb71767817a4baa994f8870ff2b0fa2 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:32:23.215889 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbbec8f-6e33-4652-b789-1f9c6f8591d9 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:32:23.216176 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbbec8f-6e33-4652-b789-1f9c6f8591d9 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:32:23.216474 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbbec8f-6e33-4652-b789-1f9c6f8591d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:32:23.236743 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-1fbbec8f-6e33-4652-b789-1f9c6f8591d9 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:32:23.237936 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-1fbbec8f-6e33-4652-b789-1f9c6f8591d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:32:23.255601 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:32:23.299115 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1137/2278] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:32:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:32:23.307091 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3337cc6-2ddc-4a7c-a5ff-cc3b91e982fc None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:32:23.307451 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3337cc6-2ddc-4a7c-a5ff-cc3b91e982fc None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:32:23.307799 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3337cc6-2ddc-4a7c-a5ff-cc3b91e982fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:32:23.329072 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-c3337cc6-2ddc-4a7c-a5ff-cc3b91e982fc None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:32:23.330267 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-c3337cc6-2ddc-4a7c-a5ff-cc3b91e982fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:32:23.338156 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1142/2279] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:32:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:32:23.343341 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbbec8f-6e33-4652-b789-1f9c6f8591d9 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:32:23.352732 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbbec8f-6e33-4652-b789-1f9c6f8591d9 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:32:23.357445 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a5cee3-6f1d-4d08-9c8d-0b54d7950a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:32:23.358045 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a5cee3-6f1d-4d08-9c8d-0b54d7950a50 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:32:23.358324 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a5cee3-6f1d-4d08-9c8d-0b54d7950a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:32:23.358639 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a5cee3-6f1d-4d08-9c8d-0b54d7950a50 admin admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:32:23.359282 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a5cee3-6f1d-4d08-9c8d-0b54d7950a50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:32:23.359585 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a5cee3-6f1d-4d08-9c8d-0b54d7950a50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:32:23.361167 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a5cee3-6f1d-4d08-9c8d-0b54d7950a50 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:32:23.372796 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1138/2280] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:32:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:32:23.378356 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3337cc6-2ddc-4a7c-a5ff-cc3b91e982fc None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:32:23.389055 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3337cc6-2ddc-4a7c-a5ff-cc3b91e982fc None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:32:23.394131 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a956bf1-5ba8-433e-a4d7-a6b47919e482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:32:23.394740 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a956bf1-5ba8-433e-a4d7-a6b47919e482 admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:32:23.395014 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a956bf1-5ba8-433e-a4d7-a6b47919e482 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:32:23.395282 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a956bf1-5ba8-433e-a4d7-a6b47919e482 admin admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:32:23.395918 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a956bf1-5ba8-433e-a4d7-a6b47919e482 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:32:23.397874 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a956bf1-5ba8-433e-a4d7-a6b47919e482 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:32:23.424931 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1143/2281] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:32:23 2022] POST /identity/v3/projects => generated 403 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:32:23.429315 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a5cee3-6f1d-4d08-9c8d-0b54d7950a50 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:32:23.439735 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a5cee3-6f1d-4d08-9c8d-0b54d7950a50 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:32:23.494993 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fededbaa-f340-42a1-9c18-20083ecb2b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:32:23.495661 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fededbaa-f340-42a1-9c18-20083ecb2b88 admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:32:23.495863 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fededbaa-f340-42a1-9c18-20083ecb2b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:32:23.496140 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fededbaa-f340-42a1-9c18-20083ecb2b88 admin admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:32:23.496782 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fededbaa-f340-42a1-9c18-20083ecb2b88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:32:23.499304 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fededbaa-f340-42a1-9c18-20083ecb2b88 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:32:23.523604 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1139/2282] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:32:23 2022] POST /identity/v3/users => generated 302 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:32:23.527670 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a956bf1-5ba8-433e-a4d7-a6b47919e482 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:32:23.537931 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a956bf1-5ba8-433e-a4d7-a6b47919e482 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:32:23.542624 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7487c01-2fa4-415b-a162-26a8126a21a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:32:23.543213 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7487c01-2fa4-415b-a162-26a8126a21a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:32:23.543492 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7487c01-2fa4-415b-a162-26a8126a21a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:32:23.543763 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7487c01-2fa4-415b-a162-26a8126a21a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:32:23.544691 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7487c01-2fa4-415b-a162-26a8126a21a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:32:23.546263 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7487c01-2fa4-415b-a162-26a8126a21a5 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:32:23.556513 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1144/2283] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:32:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:32:23.561139 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fededbaa-f340-42a1-9c18-20083ecb2b88 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:32:23.570994 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fededbaa-f340-42a1-9c18-20083ecb2b88 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:32:23.575564 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fdedce-7d53-4349-853c-9a8ebb68f5be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:32:23.577070 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fdedce-7d53-4349-853c-9a8ebb68f5be admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:32:23.577353 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fdedce-7d53-4349-853c-9a8ebb68f5be admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:32:23.577628 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fdedce-7d53-4349-853c-9a8ebb68f5be admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:32:23.578263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fdedce-7d53-4349-853c-9a8ebb68f5be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:32:23.579878 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fdedce-7d53-4349-853c-9a8ebb68f5be admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:32:23.590313 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1140/2284] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:32:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:32:23.594257 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7487c01-2fa4-415b-a162-26a8126a21a5 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:32:23.604046 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7487c01-2fa4-415b-a162-26a8126a21a5 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:32:23.608690 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5341877f-5203-4ce2-871b-59aa73982564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:32:23.609320 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5341877f-5203-4ce2-871b-59aa73982564 admin admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:32:23.609598 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5341877f-5203-4ce2-871b-59aa73982564 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:32:23.609870 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5341877f-5203-4ce2-871b-59aa73982564 admin admin] PATH_INFO: `/v3/projects/4ca583639d5a4cbb9a55662c0b660b32/users/d5f5f2e8fcf94bc3982c07e516e90643/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:32:23.624797 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5341877f-5203-4ce2-871b-59aa73982564 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ca583639d5a4cbb9a55662c0b660b32, user_id=d5f5f2e8fcf94bc3982c07e516e90643, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:32:23.627467 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5341877f-5203-4ce2-871b-59aa73982564 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:32:23.640361 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1145/2285] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:32:23 2022] PUT /identity/v3/projects/4ca583639d5a4cbb9a55662c0b660b32/users/d5f5f2e8fcf94bc3982c07e516e90643/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:32:23.651285 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6894921b-93eb-4521-ba28-a7c4bdb94162 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:32:23.651571 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6894921b-93eb-4521-ba28-a7c4bdb94162 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:32:23.651848 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6894921b-93eb-4521-ba28-a7c4bdb94162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:32:23.679236 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-6894921b-93eb-4521-ba28-a7c4bdb94162 None None] MFA Rules not processed for user `d5f5f2e8fcf94bc3982c07e516e90643`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:32:23.680479 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-6894921b-93eb-4521-ba28-a7c4bdb94162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:32:23.748426 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1141/2286] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:32:23 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:32:23.752707 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5341877f-5203-4ce2-871b-59aa73982564 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:32:23.774636 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:32:23.817746 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5341877f-5203-4ce2-871b-59aa73982564 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:32:23.822625 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd888f79-ffd6-4d0d-9077-a41fff39d38c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:32:23.823260 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd888f79-ffd6-4d0d-9077-a41fff39d38c admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:32:23.823542 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd888f79-ffd6-4d0d-9077-a41fff39d38c admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:32:23.823819 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd888f79-ffd6-4d0d-9077-a41fff39d38c admin admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:32:23.824453 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd888f79-ffd6-4d0d-9077-a41fff39d38c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:32:23.826429 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd888f79-ffd6-4d0d-9077-a41fff39d38c admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:32:23.852538 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1146/2287] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:32:23 2022] POST /identity/v3/projects => generated 405 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:32:23.856544 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6894921b-93eb-4521-ba28-a7c4bdb94162 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:32:23.867633 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6894921b-93eb-4521-ba28-a7c4bdb94162 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:32:23.880469 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:32:23.924184 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1389e046-b93e-4860-8744-6422f187724a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:32:23.924811 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1389e046-b93e-4860-8744-6422f187724a admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:32:23.925089 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1389e046-b93e-4860-8744-6422f187724a admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:32:23.925359 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1389e046-b93e-4860-8744-6422f187724a admin admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:32:23.925991 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1389e046-b93e-4860-8744-6422f187724a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:32:23.928172 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1389e046-b93e-4860-8744-6422f187724a admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:32:23.949800 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1142/2288] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:32:23 2022] POST /identity/v3/users => generated 309 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 16:32:23.953806 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd888f79-ffd6-4d0d-9077-a41fff39d38c admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:32:23.963895 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd888f79-ffd6-4d0d-9077-a41fff39d38c admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:32:23.968453 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e26b3d7-4566-41e3-9f68-7c1d15f96ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:32:23.969041 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e26b3d7-4566-41e3-9f68-7c1d15f96ea0 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:32:23.969325 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e26b3d7-4566-41e3-9f68-7c1d15f96ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:32:23.969599 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e26b3d7-4566-41e3-9f68-7c1d15f96ea0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:32:23.970235 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e26b3d7-4566-41e3-9f68-7c1d15f96ea0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:32:23.971847 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e26b3d7-4566-41e3-9f68-7c1d15f96ea0 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:32:23.982645 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1147/2289] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:32:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:32:23.986657 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1389e046-b93e-4860-8744-6422f187724a admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:32:23.996934 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1389e046-b93e-4860-8744-6422f187724a admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:32:24.002062 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf91c09-59c0-40d1-a448-191be5e9d89c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:32:24.002747 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf91c09-59c0-40d1-a448-191be5e9d89c admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:32:24.003032 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf91c09-59c0-40d1-a448-191be5e9d89c admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:32:24.003316 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf91c09-59c0-40d1-a448-191be5e9d89c admin admin] PATH_INFO: `/v3/projects/2f59693517e4447aa8f3aac4ed9f7a62/users/2f7cfca9abae4c3aaf810b663cc58c66/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:32:24.019176 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf91c09-59c0-40d1-a448-191be5e9d89c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f59693517e4447aa8f3aac4ed9f7a62, user_id=2f7cfca9abae4c3aaf810b663cc58c66, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:32:24.021823 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf91c09-59c0-40d1-a448-191be5e9d89c admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:32:24.035778 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1143/2290] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:32:23 2022] PUT /identity/v3/projects/2f59693517e4447aa8f3aac4ed9f7a62/users/2f7cfca9abae4c3aaf810b663cc58c66/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:32:24.039800 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e26b3d7-4566-41e3-9f68-7c1d15f96ea0 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:32:24.050861 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e26b3d7-4566-41e3-9f68-7c1d15f96ea0 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:32:24.110769 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4368768a-8a20-4cd6-a978-e6c43e8ad22e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:32:24.111372 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4368768a-8a20-4cd6-a978-e6c43e8ad22e admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:32:24.111651 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4368768a-8a20-4cd6-a978-e6c43e8ad22e admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:32:24.111922 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4368768a-8a20-4cd6-a978-e6c43e8ad22e admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:32:24.112552 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4368768a-8a20-4cd6-a978-e6c43e8ad22e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:32:24.114136 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4368768a-8a20-4cd6-a978-e6c43e8ad22e admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:32:24.123670 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1148/2291] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:32:24 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:32:24.127907 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf91c09-59c0-40d1-a448-191be5e9d89c admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:32:24.138109 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf91c09-59c0-40d1-a448-191be5e9d89c admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:32:24.143301 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2416aa-950a-44dd-87f7-f812696ae08a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:32:24.143942 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2416aa-950a-44dd-87f7-f812696ae08a admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:32:24.144223 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2416aa-950a-44dd-87f7-f812696ae08a admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:32:24.144500 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2416aa-950a-44dd-87f7-f812696ae08a admin admin] PATH_INFO: `/v3/projects/2f59693517e4447aa8f3aac4ed9f7a62/users/2f7cfca9abae4c3aaf810b663cc58c66/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:32:24.146416 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2416aa-950a-44dd-87f7-f812696ae08a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f59693517e4447aa8f3aac4ed9f7a62, user_id=2f7cfca9abae4c3aaf810b663cc58c66, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:32:24.149055 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2416aa-950a-44dd-87f7-f812696ae08a admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:32:24.161333 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1144/2292] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:32:24 2022] PUT /identity/v3/projects/2f59693517e4447aa8f3aac4ed9f7a62/users/2f7cfca9abae4c3aaf810b663cc58c66/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:32:24.173518 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062ad4f8-65b5-4020-8a8c-03964eb1393d None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:32:24.173831 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062ad4f8-65b5-4020-8a8c-03964eb1393d None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:32:24.174113 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062ad4f8-65b5-4020-8a8c-03964eb1393d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:32:24.203348 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-062ad4f8-65b5-4020-8a8c-03964eb1393d None None] MFA Rules not processed for user `2f7cfca9abae4c3aaf810b663cc58c66`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:32:24.204633 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-062ad4f8-65b5-4020-8a8c-03964eb1393d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:32:24.271320 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1149/2293] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:32:24 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:32:34.157210 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=36,keystone:INSERT=5 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:32:34.268306 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=92,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:32:45.728365 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2416aa-950a-44dd-87f7-f812696ae08a admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:32:45.730103 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-9b2416aa-950a-44dd-87f7-f812696ae08a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:32:45.874835 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2416aa-950a-44dd-87f7-f812696ae08a admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:32:45.880399 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22968d6b-32e8-4e98-87b7-ef3a7b2ca20d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1a985495ed48a4acc48e2e6fc7f977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:32:45.881002 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22968d6b-32e8-4e98-87b7-ef3a7b2ca20d service placement] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:32:45.881318 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22968d6b-32e8-4e98-87b7-ef3a7b2ca20d service placement] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:32:45.881637 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22968d6b-32e8-4e98-87b7-ef3a7b2ca20d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:32:45.883647 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-22968d6b-32e8-4e98-87b7-ef3a7b2ca20d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:32:45.896567 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22968d6b-32e8-4e98-87b7-ef3a7b2ca20d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:32:45.898182 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22968d6b-32e8-4e98-87b7-ef3a7b2ca20d service placement] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:32:45.908784 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1145/2294] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:32:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:32:55.904862 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=61 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:33:11.014316 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062ad4f8-65b5-4020-8a8c-03964eb1393d None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:33:11.016036 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-062ad4f8-65b5-4020-8a8c-03964eb1393d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:33:11.101396 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062ad4f8-65b5-4020-8a8c-03964eb1393d None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:33:11.105128 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97631500-6790-4f24-be6d-0183b39ba300 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:33:11.105730 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97631500-6790-4f24-be6d-0183b39ba300 service cinder] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:33:11.106000 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97631500-6790-4f24-be6d-0183b39ba300 service cinder] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:33:11.106306 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97631500-6790-4f24-be6d-0183b39ba300 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:33:11.109078 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-97631500-6790-4f24-be6d-0183b39ba300 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:33:11.188665 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97631500-6790-4f24-be6d-0183b39ba300 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:33:11.190024 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97631500-6790-4f24-be6d-0183b39ba300 service cinder] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:33:11.199619 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1150/2295] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:33:11 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:33:11.302838 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22968d6b-32e8-4e98-87b7-ef3a7b2ca20d service placement] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:33:11.316655 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22968d6b-32e8-4e98-87b7-ef3a7b2ca20d service placement] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:33:11.321266 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aef0cb6-1e88-453e-a332-a9b5ff1626cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:33:11.321879 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aef0cb6-1e88-453e-a332-a9b5ff1626cf service cinder] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:33:11.322180 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aef0cb6-1e88-453e-a332-a9b5ff1626cf service cinder] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:33:11.322484 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aef0cb6-1e88-453e-a332-a9b5ff1626cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:33:11.324373 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-6aef0cb6-1e88-453e-a332-a9b5ff1626cf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:33:11.354643 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-6aef0cb6-1e88-453e-a332-a9b5ff1626cf service cinder] Could not find project: 4eb71767817a4baa994f8870ff2b0fa2.: keystone.exception.ProjectNotFound: Could not find project: 4eb71767817a4baa994f8870ff2b0fa2. Sep 19 16:33:11.354643 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:33:11.354643 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:33:11.354643 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:33:11.354643 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:33:11.354643 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:33:11.354643 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:33:11.354643 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:33:11.354643 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:33:11.354643 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:33:11.354643 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:33:11.354643 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:33:11.354643 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Sep 19 16:33:11.354643 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Sep 19 16:33:11.354643 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Sep 19 16:33:11.356736 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Sep 19 16:33:11.356736 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Sep 19 16:33:11.356736 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Sep 19 16:33:11.356736 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:33:11.356736 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:33:11.356736 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Sep 19 16:33:11.356736 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Sep 19 16:33:11.356736 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:33:11.356736 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:33:11.356736 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:33:11.356736 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:33:11.356736 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:33:11.356736 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:33:11.356736 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:33:11.356736 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:33:11.356736 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:33:11.356736 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:33:11.358842 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:33:11.358842 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:33:11.358842 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:33:11.358842 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:33:11.358842 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Sep 19 16:33:11.358842 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Sep 19 16:33:11.358842 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Sep 19 16:33:11.358842 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application self._validate_token_resources() Sep 19 16:33:11.358842 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Sep 19 16:33:11.358842 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Sep 19 16:33:11.358842 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Sep 19 16:33:11.358842 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Sep 19 16:33:11.358842 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:33:11.358842 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:33:11.358842 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:33:11.358842 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:33:11.358842 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:33:11.360969 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:33:11.360969 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:33:11.360969 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:33:11.360969 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:33:11.360969 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:33:11.360969 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:33:11.360969 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:33:11.360969 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:33:11.360969 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:33:11.360969 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:33:11.360969 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:33:11.360969 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 16:33:11.360969 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 16:33:11.360969 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 16:33:11.360969 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 16:33:11.360969 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 16:33:11.360969 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 16:33:11.362889 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 4eb71767817a4baa994f8870ff2b0fa2. Sep 19 16:33:11.362889 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:33:11.362889 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1146/2296] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:33:11 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 54 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Sep 19 16:33:11.609893 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97631500-6790-4f24-be6d-0183b39ba300 service cinder] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:33:11.621462 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97631500-6790-4f24-be6d-0183b39ba300 service cinder] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:33:11.626755 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dfdcb4-393f-4294-93c6-6ae4240b5f1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:33:11.627434 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dfdcb4-393f-4294-93c6-6ae4240b5f1c service cinder] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:33:11.627714 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dfdcb4-393f-4294-93c6-6ae4240b5f1c service cinder] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:33:11.628043 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dfdcb4-393f-4294-93c6-6ae4240b5f1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:33:11.639931 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dfdcb4-393f-4294-93c6-6ae4240b5f1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:33:11.641363 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dfdcb4-393f-4294-93c6-6ae4240b5f1c service cinder] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:33:11.651404 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1151/2297] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:33:11 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:33:21.328514 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=6 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:33:21.646905 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=69 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:34:10.574922 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aef0cb6-1e88-453e-a332-a9b5ff1626cf service cinder] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:34:10.576533 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-6aef0cb6-1e88-453e-a332-a9b5ff1626cf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:34:10.669146 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aef0cb6-1e88-453e-a332-a9b5ff1626cf service cinder] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:34:10.674835 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a601a0c-4c13-4d62-8c32-67491f3eddbd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:34:10.675433 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a601a0c-4c13-4d62-8c32-67491f3eddbd service nova] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:34:10.675723 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a601a0c-4c13-4d62-8c32-67491f3eddbd service nova] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:34:10.676019 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a601a0c-4c13-4d62-8c32-67491f3eddbd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:34:10.678022 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-0a601a0c-4c13-4d62-8c32-67491f3eddbd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:34:10.745861 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a601a0c-4c13-4d62-8c32-67491f3eddbd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:34:10.747490 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a601a0c-4c13-4d62-8c32-67491f3eddbd service nova] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:34:10.766944 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1147/2298] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:34:10 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:34:10.864045 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dfdcb4-393f-4294-93c6-6ae4240b5f1c service cinder] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:34:10.876736 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dfdcb4-393f-4294-93c6-6ae4240b5f1c service cinder] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:34:10.882109 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845aba12-df17-49ad-8175-9e799e821023 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:34:10.882736 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845aba12-df17-49ad-8175-9e799e821023 service nova] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:34:10.883016 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845aba12-df17-49ad-8175-9e799e821023 service nova] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:34:10.883287 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845aba12-df17-49ad-8175-9e799e821023 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:34:10.884975 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-845aba12-df17-49ad-8175-9e799e821023 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:34:10.951856 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845aba12-df17-49ad-8175-9e799e821023 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:34:10.953445 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845aba12-df17-49ad-8175-9e799e821023 service nova] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:34:10.971473 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1152/2299] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:34:10 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:34:17.882612 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a601a0c-4c13-4d62-8c32-67491f3eddbd service nova] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:34:17.894000 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a601a0c-4c13-4d62-8c32-67491f3eddbd service nova] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:34:17.899512 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2817b48c-ad6c-4ddf-b784-b520a13bb39c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:34:17.900103 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2817b48c-ad6c-4ddf-b784-b520a13bb39c service nova] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:34:17.900383 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2817b48c-ad6c-4ddf-b784-b520a13bb39c service nova] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:34:17.900658 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2817b48c-ad6c-4ddf-b784-b520a13bb39c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:34:17.902357 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-2817b48c-ad6c-4ddf-b784-b520a13bb39c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:34:17.960791 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2817b48c-ad6c-4ddf-b784-b520a13bb39c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:34:17.962532 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2817b48c-ad6c-4ddf-b784-b520a13bb39c service nova] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:34:17.971856 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1148/2300] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:34:17 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:34:20.968917 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=36 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:34:27.967153 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=99 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:34:55.447577 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845aba12-df17-49ad-8175-9e799e821023 service nova] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:34:55.449076 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-845aba12-df17-49ad-8175-9e799e821023 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:34:55.518391 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845aba12-df17-49ad-8175-9e799e821023 service nova] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:34:55.522010 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c795a30-6736-4b56-bb7f-f8a2b8d07eea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:34:55.522636 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c795a30-6736-4b56-bb7f-f8a2b8d07eea service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:34:55.522918 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c795a30-6736-4b56-bb7f-f8a2b8d07eea service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:34:55.523196 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c795a30-6736-4b56-bb7f-f8a2b8d07eea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:34:55.524918 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-2c795a30-6736-4b56-bb7f-f8a2b8d07eea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:34:55.534308 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c795a30-6736-4b56-bb7f-f8a2b8d07eea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:34:55.535753 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c795a30-6736-4b56-bb7f-f8a2b8d07eea service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:34:55.543948 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1153/2301] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:34:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:35:05.540260 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=30 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:35:11.746494 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2817b48c-ad6c-4ddf-b784-b520a13bb39c service nova] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:35:11.755710 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2817b48c-ad6c-4ddf-b784-b520a13bb39c service nova] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:35:11.759339 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c966397f-0c5e-4fe9-af58-e2cdfbaa5a99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:35:11.759929 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c966397f-0c5e-4fe9-af58-e2cdfbaa5a99 service cinder] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:35:11.760210 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c966397f-0c5e-4fe9-af58-e2cdfbaa5a99 service cinder] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:35:11.760479 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c966397f-0c5e-4fe9-af58-e2cdfbaa5a99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:35:11.762199 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-c966397f-0c5e-4fe9-af58-e2cdfbaa5a99 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:35:11.776706 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c966397f-0c5e-4fe9-af58-e2cdfbaa5a99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:35:11.778120 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c966397f-0c5e-4fe9-af58-e2cdfbaa5a99 service cinder] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:35:11.789218 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1149/2302] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:35:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:35:11.884701 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c795a30-6736-4b56-bb7f-f8a2b8d07eea service neutron] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:35:11.894097 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c795a30-6736-4b56-bb7f-f8a2b8d07eea service neutron] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:35:11.897702 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d0dc8d-a396-4a53-acc1-8416d63ea312 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:35:11.898295 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d0dc8d-a396-4a53-acc1-8416d63ea312 service cinder] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:35:11.898607 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d0dc8d-a396-4a53-acc1-8416d63ea312 service cinder] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:35:11.898881 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d0dc8d-a396-4a53-acc1-8416d63ea312 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:35:11.900600 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-17d0dc8d-a396-4a53-acc1-8416d63ea312 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:35:11.959934 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d0dc8d-a396-4a53-acc1-8416d63ea312 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:35:11.961361 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d0dc8d-a396-4a53-acc1-8416d63ea312 service cinder] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:35:11.970277 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1154/2303] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:35:11 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:35:21.784307 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=6 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:35:21.966534 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=28 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:37:18.474487 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c966397f-0c5e-4fe9-af58-e2cdfbaa5a99 service cinder] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:37:18.476152 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-c966397f-0c5e-4fe9-af58-e2cdfbaa5a99 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:37:18.487117 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c966397f-0c5e-4fe9-af58-e2cdfbaa5a99 service cinder] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:37:18.491078 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188746e4-d893-4db6-ba2f-09db3ab40644 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:37:18.491687 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188746e4-d893-4db6-ba2f-09db3ab40644 service cinder] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:37:18.491968 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188746e4-d893-4db6-ba2f-09db3ab40644 service cinder] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:37:18.492242 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188746e4-d893-4db6-ba2f-09db3ab40644 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:37:18.494246 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-188746e4-d893-4db6-ba2f-09db3ab40644 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:37:18.508431 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188746e4-d893-4db6-ba2f-09db3ab40644 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:37:18.509864 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188746e4-d893-4db6-ba2f-09db3ab40644 service cinder] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:37:18.522171 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1150/2304] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:37:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:37:18.571958 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d0dc8d-a396-4a53-acc1-8416d63ea312 service cinder] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:37:18.579823 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d0dc8d-a396-4a53-acc1-8416d63ea312 service cinder] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:37:18.584004 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47b2a72-3b9c-43fb-88eb-b7438319ff94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:37:18.584638 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47b2a72-3b9c-43fb-88eb-b7438319ff94 service cinder] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:37:18.585083 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47b2a72-3b9c-43fb-88eb-b7438319ff94 service cinder] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:37:18.585390 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47b2a72-3b9c-43fb-88eb-b7438319ff94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:37:18.587934 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-c47b2a72-3b9c-43fb-88eb-b7438319ff94 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:37:18.691937 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47b2a72-3b9c-43fb-88eb-b7438319ff94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:37:18.693378 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47b2a72-3b9c-43fb-88eb-b7438319ff94 service cinder] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:37:18.724723 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1155/2305] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:37:18 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:37:28.516969 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=6 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:37:28.718228 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=36 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:37:51.720054 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188746e4-d893-4db6-ba2f-09db3ab40644 service cinder] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:37:51.731789 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188746e4-d893-4db6-ba2f-09db3ab40644 service cinder] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:37:51.737355 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7adb991-4631-4265-9e54-b9a216e14ebd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1a985495ed48a4acc48e2e6fc7f977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:37:51.737955 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7adb991-4631-4265-9e54-b9a216e14ebd service placement] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:37:51.738233 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7adb991-4631-4265-9e54-b9a216e14ebd service placement] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:37:51.738545 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7adb991-4631-4265-9e54-b9a216e14ebd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:37:51.747972 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7adb991-4631-4265-9e54-b9a216e14ebd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:37:51.749563 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7adb991-4631-4265-9e54-b9a216e14ebd service placement] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:37:51.760604 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1151/2306] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:37:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:38:01.755596 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=6 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:39:12.241740 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47b2a72-3b9c-43fb-88eb-b7438319ff94 service cinder] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:39:12.251125 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47b2a72-3b9c-43fb-88eb-b7438319ff94 service cinder] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:39:12.254844 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924d6c04-37c3-428b-a3c2-07f35f20d23c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:39:12.255577 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d6c04-37c3-428b-a3c2-07f35f20d23c service cinder] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:39:12.255868 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d6c04-37c3-428b-a3c2-07f35f20d23c service cinder] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:39:12.256140 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d6c04-37c3-428b-a3c2-07f35f20d23c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:39:12.266122 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924d6c04-37c3-428b-a3c2-07f35f20d23c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:39:12.267891 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924d6c04-37c3-428b-a3c2-07f35f20d23c service cinder] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:39:12.277434 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1156/2307] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:39:12 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:39:18.937681 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7adb991-4631-4265-9e54-b9a216e14ebd service placement] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:39:18.947003 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7adb991-4631-4265-9e54-b9a216e14ebd service placement] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:39:18.951208 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db495782-fa41-4eed-9909-d3c820f0a369 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:39:18.951796 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db495782-fa41-4eed-9909-d3c820f0a369 service cinder] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:39:18.952076 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db495782-fa41-4eed-9909-d3c820f0a369 service cinder] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:39:18.952350 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db495782-fa41-4eed-9909-d3c820f0a369 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:39:18.960858 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db495782-fa41-4eed-9909-d3c820f0a369 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:39:18.962270 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db495782-fa41-4eed-9909-d3c820f0a369 service cinder] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:39:18.990144 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1152/2308] 10.222.0.57 () {60 vars in 1396 bytes} [Mon Sep 19 16:39:18 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3260 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 16:39:19.072165 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924d6c04-37c3-428b-a3c2-07f35f20d23c service cinder] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:39:19.082039 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924d6c04-37c3-428b-a3c2-07f35f20d23c service cinder] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:39:19.087487 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09e5be9-b021-42ae-b695-b78891f3aec3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:39:19.088111 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e5be9-b021-42ae-b695-b78891f3aec3 service cinder] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:39:19.088432 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e5be9-b021-42ae-b695-b78891f3aec3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:39:19.088774 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e5be9-b021-42ae-b695-b78891f3aec3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:39:19.098040 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09e5be9-b021-42ae-b695-b78891f3aec3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:39:19.099511 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09e5be9-b021-42ae-b695-b78891f3aec3 service cinder] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:39:19.128836 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1157/2309] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:39:19 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:39:28.987229 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=13 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:39:29.125738 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=19 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:40:00.440142 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db495782-fa41-4eed-9909-d3c820f0a369 service cinder] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:40:00.449963 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db495782-fa41-4eed-9909-d3c820f0a369 service cinder] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:40:00.453873 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fa19e1-f3b7-438f-8806-5f029f20bb2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:40:00.454561 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fa19e1-f3b7-438f-8806-5f029f20bb2a service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:40:00.454837 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fa19e1-f3b7-438f-8806-5f029f20bb2a service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:40:00.455103 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fa19e1-f3b7-438f-8806-5f029f20bb2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:40:00.462752 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fa19e1-f3b7-438f-8806-5f029f20bb2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:40:00.464169 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fa19e1-f3b7-438f-8806-5f029f20bb2a service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:40:00.472497 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1153/2310] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:40:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:40:10.468881 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=6 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:41:12.339408 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09e5be9-b021-42ae-b695-b78891f3aec3 service cinder] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:41:12.349432 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09e5be9-b021-42ae-b695-b78891f3aec3 service cinder] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:41:12.354955 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58065997-d6db-4dce-a088-4ea059985ba2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:41:12.355597 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58065997-d6db-4dce-a088-4ea059985ba2 service cinder] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:41:12.356475 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58065997-d6db-4dce-a088-4ea059985ba2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:41:12.356608 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58065997-d6db-4dce-a088-4ea059985ba2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:41:12.367482 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58065997-d6db-4dce-a088-4ea059985ba2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:41:12.368906 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58065997-d6db-4dce-a088-4ea059985ba2 service cinder] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:41:12.380533 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1158/2311] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:41:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:41:12.429971 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fa19e1-f3b7-438f-8806-5f029f20bb2a service neutron] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:41:12.431372 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-d5fa19e1-f3b7-438f-8806-5f029f20bb2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:41:12.451655 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58065997-d6db-4dce-a088-4ea059985ba2 service cinder] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:41:12.460146 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58065997-d6db-4dce-a088-4ea059985ba2 service cinder] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:41:12.461575 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fa19e1-f3b7-438f-8806-5f029f20bb2a service neutron] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:41:12.463748 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24b7799-c8d6-4fec-96c4-e7c75b38ba85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:41:12.464342 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b7799-c8d6-4fec-96c4-e7c75b38ba85 service cinder] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:41:12.464623 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b7799-c8d6-4fec-96c4-e7c75b38ba85 service cinder] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:41:12.464893 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b7799-c8d6-4fec-96c4-e7c75b38ba85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:41:12.467129 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39689a1-eba3-4369-b5c7-071ae5a6835d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:41:12.467710 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39689a1-eba3-4369-b5c7-071ae5a6835d service nova] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:41:12.467984 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39689a1-eba3-4369-b5c7-071ae5a6835d service nova] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:41:12.468253 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39689a1-eba3-4369-b5c7-071ae5a6835d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:41:12.473713 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24b7799-c8d6-4fec-96c4-e7c75b38ba85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:41:12.475178 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24b7799-c8d6-4fec-96c4-e7c75b38ba85 service cinder] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:41:12.477091 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39689a1-eba3-4369-b5c7-071ae5a6835d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:41:12.478718 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39689a1-eba3-4369-b5c7-071ae5a6835d service nova] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:41:12.488180 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1159/2312] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:41:12 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 16:41:12.507676 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1154/2313] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:41:12 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:41:12.731523 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24b7799-c8d6-4fec-96c4-e7c75b38ba85 service cinder] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:41:12.745821 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24b7799-c8d6-4fec-96c4-e7c75b38ba85 service cinder] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:41:12.751562 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3279e37-384c-49e0-8390-dd9773ac52aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:41:12.752167 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3279e37-384c-49e0-8390-dd9773ac52aa service nova] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:41:12.752444 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3279e37-384c-49e0-8390-dd9773ac52aa service nova] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:41:12.752713 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3279e37-384c-49e0-8390-dd9773ac52aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:41:12.761422 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3279e37-384c-49e0-8390-dd9773ac52aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:41:12.763086 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3279e37-384c-49e0-8390-dd9773ac52aa service nova] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:41:12.773100 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1160/2314] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:41:12 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:41:22.504489 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=18 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:41:22.768762 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=18 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:42:56.709922 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39689a1-eba3-4369-b5c7-071ae5a6835d service nova] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:42:56.711518 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-f39689a1-eba3-4369-b5c7-071ae5a6835d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:42:56.838542 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39689a1-eba3-4369-b5c7-071ae5a6835d service nova] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:42:56.843767 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85369af-6395-4a0a-83fe-a9d68bb922ea service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1a985495ed48a4acc48e2e6fc7f977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:42:56.844286 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85369af-6395-4a0a-83fe-a9d68bb922ea service placement] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:42:56.844563 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85369af-6395-4a0a-83fe-a9d68bb922ea service placement] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:42:56.844833 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85369af-6395-4a0a-83fe-a9d68bb922ea service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:42:56.846680 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-e85369af-6395-4a0a-83fe-a9d68bb922ea service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:42:56.857896 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85369af-6395-4a0a-83fe-a9d68bb922ea service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:42:56.859608 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85369af-6395-4a0a-83fe-a9d68bb922ea service placement] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:42:56.869508 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1155/2315] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:42:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:43:06.865367 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=57 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:43:12.644994 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3279e37-384c-49e0-8390-dd9773ac52aa service nova] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:43:12.646763 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-a3279e37-384c-49e0-8390-dd9773ac52aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:43:12.759139 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3279e37-384c-49e0-8390-dd9773ac52aa service nova] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:43:12.763624 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523af9c5-b956-41fc-8eb4-67f334c16b88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:43:12.764261 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523af9c5-b956-41fc-8eb4-67f334c16b88 service cinder] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:43:12.764542 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523af9c5-b956-41fc-8eb4-67f334c16b88 service cinder] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:43:12.764815 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523af9c5-b956-41fc-8eb4-67f334c16b88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:43:12.777504 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523af9c5-b956-41fc-8eb4-67f334c16b88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:43:12.778901 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523af9c5-b956-41fc-8eb4-67f334c16b88 service cinder] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:43:12.790853 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1161/2316] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:43:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:43:19.458222 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85369af-6395-4a0a-83fe-a9d68bb922ea service placement] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:43:19.475231 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85369af-6395-4a0a-83fe-a9d68bb922ea service placement] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:43:19.486873 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212e527a-fc94-4e32-ab70-42ac030ea13b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:43:19.488682 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e527a-fc94-4e32-ab70-42ac030ea13b service nova] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:43:19.489128 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e527a-fc94-4e32-ab70-42ac030ea13b service nova] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:43:19.489748 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e527a-fc94-4e32-ab70-42ac030ea13b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:43:19.497401 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523af9c5-b956-41fc-8eb4-67f334c16b88 service cinder] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:43:19.499160 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-523af9c5-b956-41fc-8eb4-67f334c16b88 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:43:19.503759 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212e527a-fc94-4e32-ab70-42ac030ea13b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:43:19.506155 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212e527a-fc94-4e32-ab70-42ac030ea13b service nova] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:43:19.517266 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1156/2317] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:43:19 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:43:19.668991 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523af9c5-b956-41fc-8eb4-67f334c16b88 service cinder] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:43:19.674564 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21430c15-5e62-4fbb-8d73-d95124732f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:43:19.675260 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21430c15-5e62-4fbb-8d73-d95124732f0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:43:19.675539 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21430c15-5e62-4fbb-8d73-d95124732f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:43:19.675844 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21430c15-5e62-4fbb-8d73-d95124732f0e admin admin] PATH_INFO: `/v3/users/d5f5f2e8fcf94bc3982c07e516e90643` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:43:19.692430 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21430c15-5e62-4fbb-8d73-d95124732f0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5f5f2e8fcf94bc3982c07e516e90643)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:43:19.694298 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21430c15-5e62-4fbb-8d73-d95124732f0e admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:43:19.727299 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:43:19.761182 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-21430c15-5e62-4fbb-8d73-d95124732f0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5f5f2e8fcf94bc3982c07e516e90643'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:43:19.762329 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-21430c15-5e62-4fbb-8d73-d95124732f0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5f5f2e8fcf94bc3982c07e516e90643'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:43:19.770706 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-21430c15-5e62-4fbb-8d73-d95124732f0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5f5f2e8fcf94bc3982c07e516e90643'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:43:19.789062 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-21430c15-5e62-4fbb-8d73-d95124732f0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5f5f2e8fcf94bc3982c07e516e90643'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:43:19.789062 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-21430c15-5e62-4fbb-8d73-d95124732f0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5f5f2e8fcf94bc3982c07e516e90643'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:43:19.805476 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1162/2318] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:43:19 2022] DELETE /identity/v3/users/d5f5f2e8fcf94bc3982c07e516e90643 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 16:43:19.810158 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aaed6d-4076-41f2-897f-8970477a1b6e None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:43:19.810358 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aaed6d-4076-41f2-897f-8970477a1b6e None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:43:19.810656 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aaed6d-4076-41f2-897f-8970477a1b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:43:19.842591 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-b5aaed6d-4076-41f2-897f-8970477a1b6e None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:43:19.844463 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-b5aaed6d-4076-41f2-897f-8970477a1b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:43:19.912750 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1157/2319] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:43:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:43:19.923295 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21430c15-5e62-4fbb-8d73-d95124732f0e admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:43:19.924839 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-21430c15-5e62-4fbb-8d73-d95124732f0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:43:20.002321 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21430c15-5e62-4fbb-8d73-d95124732f0e admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:43:20.006252 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd9c24b-7de6-4fd7-91d9-79114f573ada service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:43:20.006917 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9c24b-7de6-4fd7-91d9-79114f573ada service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:43:20.007195 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9c24b-7de6-4fd7-91d9-79114f573ada service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:43:20.007463 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9c24b-7de6-4fd7-91d9-79114f573ada service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:43:20.016966 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd9c24b-7de6-4fd7-91d9-79114f573ada service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:43:20.018437 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd9c24b-7de6-4fd7-91d9-79114f573ada service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:43:20.029364 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1163/2320] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:43:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:43:20.359914 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aaed6d-4076-41f2-897f-8970477a1b6e None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:43:20.361415 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-b5aaed6d-4076-41f2-897f-8970477a1b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:43:20.375738 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aaed6d-4076-41f2-897f-8970477a1b6e None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:43:20.381623 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12920e15-a047-4328-b506-e85d556b3ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:43:20.382280 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12920e15-a047-4328-b506-e85d556b3ae0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:43:20.382594 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12920e15-a047-4328-b506-e85d556b3ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:43:20.382874 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12920e15-a047-4328-b506-e85d556b3ae0 admin admin] PATH_INFO: `/v3/projects/4ca583639d5a4cbb9a55662c0b660b32` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:43:20.399007 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12920e15-a047-4328-b506-e85d556b3ae0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ca583639d5a4cbb9a55662c0b660b32)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:43:20.400881 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12920e15-a047-4328-b506-e85d556b3ae0 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:43:20.459969 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-12920e15-a047-4328-b506-e85d556b3ae0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ca583639d5a4cbb9a55662c0b660b32', 'request_id': 'req-12920e15-a047-4328-b506-e85d556b3ae0'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:43:20.469493 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-12920e15-a047-4328-b506-e85d556b3ae0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ca583639d5a4cbb9a55662c0b660b32', 'request_id': 'req-12920e15-a047-4328-b506-e85d556b3ae0'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:43:20.476127 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-12920e15-a047-4328-b506-e85d556b3ae0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ca583639d5a4cbb9a55662c0b660b32', 'request_id': 'req-12920e15-a047-4328-b506-e85d556b3ae0'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:43:20.493511 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-12920e15-a047-4328-b506-e85d556b3ae0 admin admin] The token cache is being invalidate because project 4ca583639d5a4cbb9a55662c0b660b32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81618) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:43:20.493903 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-12920e15-a047-4328-b506-e85d556b3ae0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:43:20.494706 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-12920e15-a047-4328-b506-e85d556b3ae0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:43:20.496613 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1158/2321] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:43:20 2022] DELETE /identity/v3/projects/4ca583639d5a4cbb9a55662c0b660b32 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:43:20.500943 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd9c24b-7de6-4fd7-91d9-79114f573ada service neutron] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:43:20.511131 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd9c24b-7de6-4fd7-91d9-79114f573ada service neutron] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:43:20.584059 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2395b4c-e7c1-4fc4-bebc-e9ebd8d88711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:43:20.584707 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2395b4c-e7c1-4fc4-bebc-e9ebd8d88711 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:43:20.584995 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2395b4c-e7c1-4fc4-bebc-e9ebd8d88711 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:43:20.585270 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2395b4c-e7c1-4fc4-bebc-e9ebd8d88711 admin admin] PATH_INFO: `/v3/users/2f7cfca9abae4c3aaf810b663cc58c66` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:43:20.601777 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2395b4c-e7c1-4fc4-bebc-e9ebd8d88711 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f7cfca9abae4c3aaf810b663cc58c66)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:43:20.603763 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2395b4c-e7c1-4fc4-bebc-e9ebd8d88711 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:43:20.634621 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:43:20.657352 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-f2395b4c-e7c1-4fc4-bebc-e9ebd8d88711 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f7cfca9abae4c3aaf810b663cc58c66'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:43:20.658012 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-f2395b4c-e7c1-4fc4-bebc-e9ebd8d88711 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f7cfca9abae4c3aaf810b663cc58c66'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:43:20.665830 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-f2395b4c-e7c1-4fc4-bebc-e9ebd8d88711 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f7cfca9abae4c3aaf810b663cc58c66'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:43:20.679661 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-f2395b4c-e7c1-4fc4-bebc-e9ebd8d88711 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f7cfca9abae4c3aaf810b663cc58c66'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:43:20.680582 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-f2395b4c-e7c1-4fc4-bebc-e9ebd8d88711 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f7cfca9abae4c3aaf810b663cc58c66'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:43:20.696688 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1164/2322] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:43:20 2022] DELETE /identity/v3/users/2f7cfca9abae4c3aaf810b663cc58c66 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 16:43:21.017319 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12920e15-a047-4328-b506-e85d556b3ae0 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:43:21.019068 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-12920e15-a047-4328-b506-e85d556b3ae0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:43:21.052605 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12920e15-a047-4328-b506-e85d556b3ae0 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:43:21.131428 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:43:21.147757 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c0df59-47d7-448d-8b15-93f0ef9788f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:43:21.148787 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0df59-47d7-448d-8b15-93f0ef9788f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:43:21.149064 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0df59-47d7-448d-8b15-93f0ef9788f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:43:21.149336 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0df59-47d7-448d-8b15-93f0ef9788f9 admin admin] PATH_INFO: `/v3/projects/2f59693517e4447aa8f3aac4ed9f7a62` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:43:21.163765 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c0df59-47d7-448d-8b15-93f0ef9788f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f59693517e4447aa8f3aac4ed9f7a62)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:43:21.167926 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c0df59-47d7-448d-8b15-93f0ef9788f9 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:43:21.198407 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2395b4c-e7c1-4fc4-bebc-e9ebd8d88711 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:43:21.198407 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-f2395b4c-e7c1-4fc4-bebc-e9ebd8d88711 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:43:21.268185 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2395b4c-e7c1-4fc4-bebc-e9ebd8d88711 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:43:21.268451 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-30c0df59-47d7-448d-8b15-93f0ef9788f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f59693517e4447aa8f3aac4ed9f7a62', 'request_id': 'req-30c0df59-47d7-448d-8b15-93f0ef9788f9'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:43:21.276655 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-30c0df59-47d7-448d-8b15-93f0ef9788f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f59693517e4447aa8f3aac4ed9f7a62', 'request_id': 'req-30c0df59-47d7-448d-8b15-93f0ef9788f9'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:43:21.282813 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-30c0df59-47d7-448d-8b15-93f0ef9788f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f59693517e4447aa8f3aac4ed9f7a62', 'request_id': 'req-30c0df59-47d7-448d-8b15-93f0ef9788f9'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:43:21.300683 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-30c0df59-47d7-448d-8b15-93f0ef9788f9 admin admin] The token cache is being invalidate because project 2f59693517e4447aa8f3aac4ed9f7a62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81618) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:43:21.300992 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-30c0df59-47d7-448d-8b15-93f0ef9788f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:43:21.301729 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-30c0df59-47d7-448d-8b15-93f0ef9788f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:43:21.304284 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1159/2323] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:43:21 2022] DELETE /identity/v3/projects/2f59693517e4447aa8f3aac4ed9f7a62 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:43:21.309716 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29193606-76bf-4d35-afd2-23325861af8b None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:43:21.310005 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29193606-76bf-4d35-afd2-23325861af8b None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:43:21.310285 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29193606-76bf-4d35-afd2-23325861af8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:43:21.329437 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692469ac-5f9a-458b-855c-505513c9e463 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:43:21.329996 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692469ac-5f9a-458b-855c-505513c9e463 service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:43:21.330277 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692469ac-5f9a-458b-855c-505513c9e463 service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:43:21.330600 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692469ac-5f9a-458b-855c-505513c9e463 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:43:21.332708 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-692469ac-5f9a-458b-855c-505513c9e463 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:43:21.376544 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-29193606-76bf-4d35-afd2-23325861af8b None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:43:21.377879 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-29193606-76bf-4d35-afd2-23325861af8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:43:21.421210 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692469ac-5f9a-458b-855c-505513c9e463 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:43:21.423535 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692469ac-5f9a-458b-855c-505513c9e463 service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:43:21.436616 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1165/2324] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:43:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:43:21.470576 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1160/2325] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:43:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:43:21.478423 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3f0cb-9e12-435c-9eed-bff0ca1649fc None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:43:21.479536 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3f0cb-9e12-435c-9eed-bff0ca1649fc None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:43:21.480134 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3f0cb-9e12-435c-9eed-bff0ca1649fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:43:21.513097 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-30c3f0cb-9e12-435c-9eed-bff0ca1649fc None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:43:21.514323 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-30c3f0cb-9e12-435c-9eed-bff0ca1649fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:43:21.523169 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1166/2326] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:43:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:43:21.527868 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29193606-76bf-4d35-afd2-23325861af8b None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:43:21.543027 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29193606-76bf-4d35-afd2-23325861af8b None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:43:21.573304 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b388423d-8e71-438c-b31b-ede75891d2d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:43:21.573938 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b388423d-8e71-438c-b31b-ede75891d2d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:43:21.574219 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b388423d-8e71-438c-b31b-ede75891d2d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:43:21.574535 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b388423d-8e71-438c-b31b-ede75891d2d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:43:21.575210 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b388423d-8e71-438c-b31b-ede75891d2d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:43:21.575519 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b388423d-8e71-438c-b31b-ede75891d2d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:43:21.577158 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b388423d-8e71-438c-b31b-ede75891d2d1 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:43:21.593507 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1161/2327] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:43:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:43:21.598599 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c3f0cb-9e12-435c-9eed-bff0ca1649fc None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:43:21.607002 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:43:21.615538 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c3f0cb-9e12-435c-9eed-bff0ca1649fc None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:43:21.623388 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee6893-03e2-4f58-b9cb-554d035be18f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:43:21.624158 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee6893-03e2-4f58-b9cb-554d035be18f admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:43:21.624660 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee6893-03e2-4f58-b9cb-554d035be18f admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:43:21.625039 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee6893-03e2-4f58-b9cb-554d035be18f admin admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:43:21.625894 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fee6893-03e2-4f58-b9cb-554d035be18f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:43:21.628061 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fee6893-03e2-4f58-b9cb-554d035be18f admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:43:21.684942 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1167/2328] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:43:21 2022] POST /identity/v3/projects => generated 403 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:43:21.689981 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b388423d-8e71-438c-b31b-ede75891d2d1 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:43:21.701451 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b388423d-8e71-438c-b31b-ede75891d2d1 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:43:21.741840 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:43:21.786822 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b8b5e4-fe38-41c7-b692-d7f3b47b8e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:43:21.787449 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b8b5e4-fe38-41c7-b692-d7f3b47b8e25 admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:43:21.787726 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b8b5e4-fe38-41c7-b692-d7f3b47b8e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:43:21.787995 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b8b5e4-fe38-41c7-b692-d7f3b47b8e25 admin admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:43:21.788660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b8b5e4-fe38-41c7-b692-d7f3b47b8e25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:43:21.791175 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b8b5e4-fe38-41c7-b692-d7f3b47b8e25 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:43:21.819207 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1162/2329] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:43:21 2022] POST /identity/v3/users => generated 302 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 16:43:21.823685 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee6893-03e2-4f58-b9cb-554d035be18f admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:43:21.836233 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee6893-03e2-4f58-b9cb-554d035be18f admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:43:21.841666 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40f8670-ee7d-46bb-805d-2efac7429168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:43:21.842301 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40f8670-ee7d-46bb-805d-2efac7429168 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:43:21.843233 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40f8670-ee7d-46bb-805d-2efac7429168 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:43:21.843233 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40f8670-ee7d-46bb-805d-2efac7429168 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:43:21.843557 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40f8670-ee7d-46bb-805d-2efac7429168 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:43:21.846688 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40f8670-ee7d-46bb-805d-2efac7429168 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:43:21.858907 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1168/2330] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:43:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:43:21.863103 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b8b5e4-fe38-41c7-b692-d7f3b47b8e25 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:43:21.877550 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b8b5e4-fe38-41c7-b692-d7f3b47b8e25 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:43:21.884041 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b90395-b2b7-4fa2-b9a2-6a86e7dc3634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:43:21.884666 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b90395-b2b7-4fa2-b9a2-6a86e7dc3634 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:43:21.884947 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b90395-b2b7-4fa2-b9a2-6a86e7dc3634 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:43:21.885220 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b90395-b2b7-4fa2-b9a2-6a86e7dc3634 admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:43:21.885986 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b90395-b2b7-4fa2-b9a2-6a86e7dc3634 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:43:21.888071 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b90395-b2b7-4fa2-b9a2-6a86e7dc3634 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:43:21.903208 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1163/2331] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:43:21 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:43:21.907985 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40f8670-ee7d-46bb-805d-2efac7429168 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:43:21.925161 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40f8670-ee7d-46bb-805d-2efac7429168 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:43:21.932619 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798391b6-ae4a-4052-b3b5-5e3ea9835604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:43:21.933321 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798391b6-ae4a-4052-b3b5-5e3ea9835604 admin admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:43:21.933925 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798391b6-ae4a-4052-b3b5-5e3ea9835604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:43:21.933925 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798391b6-ae4a-4052-b3b5-5e3ea9835604 admin admin] PATH_INFO: `/v3/projects/5e22167ee67c4117acea3422bb9ae5fc/users/89fd14c7ee004d3fa7f5969f2c2e8015/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:43:21.952092 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798391b6-ae4a-4052-b3b5-5e3ea9835604 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e22167ee67c4117acea3422bb9ae5fc, user_id=89fd14c7ee004d3fa7f5969f2c2e8015, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:43:21.955351 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798391b6-ae4a-4052-b3b5-5e3ea9835604 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:43:21.971256 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1169/2332] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:43:21 2022] PUT /identity/v3/projects/5e22167ee67c4117acea3422bb9ae5fc/users/89fd14c7ee004d3fa7f5969f2c2e8015/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:43:21.977541 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a27c41-1c75-4ddb-885d-c19eac4efaa7 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:43:21.978055 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a27c41-1c75-4ddb-885d-c19eac4efaa7 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:43:21.978543 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a27c41-1c75-4ddb-885d-c19eac4efaa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:43:22.003766 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-62a27c41-1c75-4ddb-885d-c19eac4efaa7 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:43:22.005046 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-62a27c41-1c75-4ddb-885d-c19eac4efaa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:43:22.092232 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1164/2333] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:43:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:43:22.105316 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798391b6-ae4a-4052-b3b5-5e3ea9835604 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:43:22.108238 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-798391b6-ae4a-4052-b3b5-5e3ea9835604 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:43:22.193264 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798391b6-ae4a-4052-b3b5-5e3ea9835604 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:43:22.198358 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa667f20-2158-4b7d-ab20-a6411e211806 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:43:22.199382 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa667f20-2158-4b7d-ab20-a6411e211806 service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:43:22.199774 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa667f20-2158-4b7d-ab20-a6411e211806 service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:43:22.200289 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa667f20-2158-4b7d-ab20-a6411e211806 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:43:22.212928 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa667f20-2158-4b7d-ab20-a6411e211806 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:43:22.214764 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa667f20-2158-4b7d-ab20-a6411e211806 service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:43:22.226289 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1170/2334] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:43:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:43:23.918591 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e215dce5-7030-49c0-8d30-b47e32471de3 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:43:23.919104 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e215dce5-7030-49c0-8d30-b47e32471de3 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:43:23.919580 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e215dce5-7030-49c0-8d30-b47e32471de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:43:23.976859 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-e215dce5-7030-49c0-8d30-b47e32471de3 None None] MFA Rules not processed for user `89fd14c7ee004d3fa7f5969f2c2e8015`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:43:23.978481 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-e215dce5-7030-49c0-8d30-b47e32471de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:43:24.137891 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1165/2335] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:43:23 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:43:24.142627 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa667f20-2158-4b7d-ab20-a6411e211806 service neutron] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:43:24.154587 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa667f20-2158-4b7d-ab20-a6411e211806 service neutron] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:43:24.162793 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec116c7e-784f-43b5-9cab-2e075bc8f501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:43:24.163428 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec116c7e-784f-43b5-9cab-2e075bc8f501 admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:43:24.163696 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec116c7e-784f-43b5-9cab-2e075bc8f501 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:43:24.164161 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec116c7e-784f-43b5-9cab-2e075bc8f501 admin admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:43:24.165007 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec116c7e-784f-43b5-9cab-2e075bc8f501 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:43:24.167171 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec116c7e-784f-43b5-9cab-2e075bc8f501 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:43:24.213956 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1171/2336] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:43:24 2022] POST /identity/v3/projects => generated 403 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:43:24.217844 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e215dce5-7030-49c0-8d30-b47e32471de3 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:43:24.230902 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e215dce5-7030-49c0-8d30-b47e32471de3 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:43:24.290763 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:43:24.309800 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839398d5-c5fd-471d-baff-2e3f688c1c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:43:24.310455 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839398d5-c5fd-471d-baff-2e3f688c1c40 admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:43:24.310741 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839398d5-c5fd-471d-baff-2e3f688c1c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:43:24.311017 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839398d5-c5fd-471d-baff-2e3f688c1c40 admin admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:43:24.311682 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839398d5-c5fd-471d-baff-2e3f688c1c40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:43:24.313712 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839398d5-c5fd-471d-baff-2e3f688c1c40 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:43:24.360352 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1166/2337] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:43:24 2022] POST /identity/v3/users => generated 308 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 16:43:24.362550 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec116c7e-784f-43b5-9cab-2e075bc8f501 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:43:24.388902 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec116c7e-784f-43b5-9cab-2e075bc8f501 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:43:24.396636 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec89938-de68-4793-9c09-7f2872c0ea47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:43:24.397554 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec89938-de68-4793-9c09-7f2872c0ea47 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:43:24.397841 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec89938-de68-4793-9c09-7f2872c0ea47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:43:24.398134 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec89938-de68-4793-9c09-7f2872c0ea47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:43:24.398824 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec89938-de68-4793-9c09-7f2872c0ea47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:43:24.400521 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec89938-de68-4793-9c09-7f2872c0ea47 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:43:24.416321 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1172/2338] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:43:24 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:43:24.421237 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839398d5-c5fd-471d-baff-2e3f688c1c40 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:43:24.440799 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839398d5-c5fd-471d-baff-2e3f688c1c40 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:43:24.447325 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98fffca-77ae-4f9c-8920-5842e7100ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:43:24.448633 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98fffca-77ae-4f9c-8920-5842e7100ab1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:43:24.449220 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98fffca-77ae-4f9c-8920-5842e7100ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:43:24.449763 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98fffca-77ae-4f9c-8920-5842e7100ab1 admin admin] PATH_INFO: `/v3/projects/40395b3a79a444c9af480a4e8f5b7731/users/35f476aac39042d2b638c8ce68cb0866/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:43:24.471271 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98fffca-77ae-4f9c-8920-5842e7100ab1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40395b3a79a444c9af480a4e8f5b7731, user_id=35f476aac39042d2b638c8ce68cb0866, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:43:24.474726 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98fffca-77ae-4f9c-8920-5842e7100ab1 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:43:24.499551 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1167/2339] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:43:24 2022] PUT /identity/v3/projects/40395b3a79a444c9af480a4e8f5b7731/users/35f476aac39042d2b638c8ce68cb0866/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:43:24.501917 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec89938-de68-4793-9c09-7f2872c0ea47 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:43:24.523866 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec89938-de68-4793-9c09-7f2872c0ea47 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:43:24.634480 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea0850c-88a7-4f79-83f8-f0b17db32344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:43:24.636185 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0850c-88a7-4f79-83f8-f0b17db32344 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:43:24.636185 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0850c-88a7-4f79-83f8-f0b17db32344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:43:24.636185 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0850c-88a7-4f79-83f8-f0b17db32344 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:43:24.636790 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea0850c-88a7-4f79-83f8-f0b17db32344 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:43:24.638660 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea0850c-88a7-4f79-83f8-f0b17db32344 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:43:24.650909 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1173/2340] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:43:24 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:43:24.656795 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98fffca-77ae-4f9c-8920-5842e7100ab1 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:43:24.667998 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98fffca-77ae-4f9c-8920-5842e7100ab1 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:43:24.673864 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1bbbbb-62dd-441d-87b1-789fbd270fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:43:24.674895 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1bbbbb-62dd-441d-87b1-789fbd270fe3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:43:24.677048 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1bbbbb-62dd-441d-87b1-789fbd270fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:43:24.677048 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1bbbbb-62dd-441d-87b1-789fbd270fe3 admin admin] PATH_INFO: `/v3/projects/40395b3a79a444c9af480a4e8f5b7731/users/35f476aac39042d2b638c8ce68cb0866/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:43:24.678479 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1bbbbb-62dd-441d-87b1-789fbd270fe3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40395b3a79a444c9af480a4e8f5b7731, user_id=35f476aac39042d2b638c8ce68cb0866, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:43:24.681387 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1bbbbb-62dd-441d-87b1-789fbd270fe3 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:43:24.698846 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1168/2341] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:43:24 2022] PUT /identity/v3/projects/40395b3a79a444c9af480a4e8f5b7731/users/35f476aac39042d2b638c8ce68cb0866/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:43:25.398415 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e4d787-5f61-4f6a-a713-87430c5b3451 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:43:25.398992 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e4d787-5f61-4f6a-a713-87430c5b3451 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:43:25.399377 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e4d787-5f61-4f6a-a713-87430c5b3451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:43:25.410191 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:43:25.454280 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-69e4d787-5f61-4f6a-a713-87430c5b3451 None None] MFA Rules not processed for user `bd190d8e252342ca9232f7a0c966c715`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:43:25.455596 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-69e4d787-5f61-4f6a-a713-87430c5b3451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:43:25.535872 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1174/2342] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:43:25 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:43:26.036314 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cc310a-f837-4b90-b5b2-051d7c0ac277 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:43:26.037886 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cc310a-f837-4b90-b5b2-051d7c0ac277 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:43:26.038693 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cc310a-f837-4b90-b5b2-051d7c0ac277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:43:26.104328 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-f2cc310a-f837-4b90-b5b2-051d7c0ac277 None None] MFA Rules not processed for user `35f476aac39042d2b638c8ce68cb0866`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:43:26.105155 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-f2cc310a-f837-4b90-b5b2-051d7c0ac277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:43:26.159500 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e4d787-5f61-4f6a-a713-87430c5b3451 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:43:26.283205 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1169/2343] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:43:26 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:43:26.303808 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e4d787-5f61-4f6a-a713-87430c5b3451 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:43:26.328776 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4892bd80-9c72-4bc3-96ce-127f529b54d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:43:26.330118 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4892bd80-9c72-4bc3-96ce-127f529b54d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:43:26.330118 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4892bd80-9c72-4bc3-96ce-127f529b54d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:43:26.330616 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4892bd80-9c72-4bc3-96ce-127f529b54d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:43:26.339999 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-4892bd80-9c72-4bc3-96ce-127f529b54d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:43:26.353994 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4892bd80-9c72-4bc3-96ce-127f529b54d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:43:26.355821 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4892bd80-9c72-4bc3-96ce-127f529b54d3 service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:43:26.368645 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1175/2344] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:43:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 16:43:27.000534 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cc310a-f837-4b90-b5b2-051d7c0ac277 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:43:27.002835 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-f2cc310a-f837-4b90-b5b2-051d7c0ac277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:43:27.088098 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4892bd80-9c72-4bc3-96ce-127f529b54d3 service neutron] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:43:27.090121 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-4892bd80-9c72-4bc3-96ce-127f529b54d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:43:27.133314 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cc310a-f837-4b90-b5b2-051d7c0ac277 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:43:27.144303 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfc67f8-36bc-4f6a-8e76-b6a8fc38b524 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1a985495ed48a4acc48e2e6fc7f977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:43:27.145266 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc67f8-36bc-4f6a-8e76-b6a8fc38b524 service placement] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:43:27.145725 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc67f8-36bc-4f6a-8e76-b6a8fc38b524 service placement] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:43:27.146188 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc67f8-36bc-4f6a-8e76-b6a8fc38b524 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:43:27.151066 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-6bfc67f8-36bc-4f6a-8e76-b6a8fc38b524 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:43:27.172578 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfc67f8-36bc-4f6a-8e76-b6a8fc38b524 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:43:27.176583 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfc67f8-36bc-4f6a-8e76-b6a8fc38b524 service placement] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:43:27.199273 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1170/2345] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:43:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:43:27.199600 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4892bd80-9c72-4bc3-96ce-127f529b54d3 service neutron] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:43:27.217856 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6878e42c-6f49-4339-aeeb-f346d1688f3a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1a985495ed48a4acc48e2e6fc7f977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:43:27.219349 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6878e42c-6f49-4339-aeeb-f346d1688f3a service placement] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:43:27.219873 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6878e42c-6f49-4339-aeeb-f346d1688f3a service placement] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:43:27.220565 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6878e42c-6f49-4339-aeeb-f346d1688f3a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:43:27.223718 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-6878e42c-6f49-4339-aeeb-f346d1688f3a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:43:27.243976 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6878e42c-6f49-4339-aeeb-f346d1688f3a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:43:27.245756 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6878e42c-6f49-4339-aeeb-f346d1688f3a service placement] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:43:27.261055 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1176/2346] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:43:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:43:27.631652 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfc67f8-36bc-4f6a-8e76-b6a8fc38b524 service placement] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:43:27.643946 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfc67f8-36bc-4f6a-8e76-b6a8fc38b524 service placement] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:43:27.657902 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd34667-4402-47db-9dcb-8a5c80cf51ac service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1a985495ed48a4acc48e2e6fc7f977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:43:27.658780 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd34667-4402-47db-9dcb-8a5c80cf51ac service placement] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:43:27.659248 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd34667-4402-47db-9dcb-8a5c80cf51ac service placement] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:43:27.659667 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd34667-4402-47db-9dcb-8a5c80cf51ac service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:43:27.661913 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-6bd34667-4402-47db-9dcb-8a5c80cf51ac service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:43:27.676573 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd34667-4402-47db-9dcb-8a5c80cf51ac service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:43:27.679013 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd34667-4402-47db-9dcb-8a5c80cf51ac service placement] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:43:27.689624 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:43:27.702198 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1171/2347] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:43:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:43:28.629858 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6878e42c-6f49-4339-aeeb-f346d1688f3a service placement] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:43:28.691173 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:43:28.798695 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6878e42c-6f49-4339-aeeb-f346d1688f3a service placement] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:43:28.802290 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491aff11-b3db-4b50-8548-887237d397a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:43:28.803229 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491aff11-b3db-4b50-8548-887237d397a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:43:28.803877 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491aff11-b3db-4b50-8548-887237d397a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:43:28.804630 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491aff11-b3db-4b50-8548-887237d397a9 admin admin] PATH_INFO: `/v3/users/89fd14c7ee004d3fa7f5969f2c2e8015` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:43:28.806600 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491aff11-b3db-4b50-8548-887237d397a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89fd14c7ee004d3fa7f5969f2c2e8015)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:43:28.808813 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491aff11-b3db-4b50-8548-887237d397a9 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:43:28.900763 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-491aff11-b3db-4b50-8548-887237d397a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89fd14c7ee004d3fa7f5969f2c2e8015'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:43:28.900763 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-491aff11-b3db-4b50-8548-887237d397a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89fd14c7ee004d3fa7f5969f2c2e8015'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:43:28.914580 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-491aff11-b3db-4b50-8548-887237d397a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89fd14c7ee004d3fa7f5969f2c2e8015'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:43:28.943243 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-491aff11-b3db-4b50-8548-887237d397a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89fd14c7ee004d3fa7f5969f2c2e8015'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:43:28.948517 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-491aff11-b3db-4b50-8548-887237d397a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89fd14c7ee004d3fa7f5969f2c2e8015'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:43:28.965885 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1177/2348] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:43:28 2022] DELETE /identity/v3/users/89fd14c7ee004d3fa7f5969f2c2e8015 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:43:29.192850 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd34667-4402-47db-9dcb-8a5c80cf51ac service placement] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:43:29.222412 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd34667-4402-47db-9dcb-8a5c80cf51ac service placement] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:43:29.308425 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b118b45-0ea5-488a-93d9-3408a87fdf41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:43:29.309135 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b118b45-0ea5-488a-93d9-3408a87fdf41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:43:29.309416 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b118b45-0ea5-488a-93d9-3408a87fdf41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:43:29.309698 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b118b45-0ea5-488a-93d9-3408a87fdf41 admin admin] PATH_INFO: `/v3/projects/5e22167ee67c4117acea3422bb9ae5fc` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:43:29.311060 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b118b45-0ea5-488a-93d9-3408a87fdf41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e22167ee67c4117acea3422bb9ae5fc)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:43:29.313671 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b118b45-0ea5-488a-93d9-3408a87fdf41 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:43:29.380110 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-4b118b45-0ea5-488a-93d9-3408a87fdf41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e22167ee67c4117acea3422bb9ae5fc', 'request_id': 'req-4b118b45-0ea5-488a-93d9-3408a87fdf41'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:43:29.387561 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-4b118b45-0ea5-488a-93d9-3408a87fdf41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e22167ee67c4117acea3422bb9ae5fc', 'request_id': 'req-4b118b45-0ea5-488a-93d9-3408a87fdf41'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:43:29.394155 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-4b118b45-0ea5-488a-93d9-3408a87fdf41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e22167ee67c4117acea3422bb9ae5fc', 'request_id': 'req-4b118b45-0ea5-488a-93d9-3408a87fdf41'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:43:29.413285 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-4b118b45-0ea5-488a-93d9-3408a87fdf41 admin admin] The token cache is being invalidate because project 5e22167ee67c4117acea3422bb9ae5fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81618) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:43:29.414223 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-4b118b45-0ea5-488a-93d9-3408a87fdf41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:43:29.415436 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-4b118b45-0ea5-488a-93d9-3408a87fdf41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:43:29.417358 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1172/2349] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:43:29 2022] DELETE /identity/v3/projects/5e22167ee67c4117acea3422bb9ae5fc => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:43:29.422816 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491aff11-b3db-4b50-8548-887237d397a9 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:43:29.425292 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-491aff11-b3db-4b50-8548-887237d397a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:43:29.449169 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491aff11-b3db-4b50-8548-887237d397a9 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:43:29.527852 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db77e18d-f447-4c86-b12b-d7a592adde89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:43:29.528545 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db77e18d-f447-4c86-b12b-d7a592adde89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:43:29.528820 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db77e18d-f447-4c86-b12b-d7a592adde89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:43:29.529087 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db77e18d-f447-4c86-b12b-d7a592adde89 admin admin] PATH_INFO: `/v3/users/35f476aac39042d2b638c8ce68cb0866` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:43:29.530413 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db77e18d-f447-4c86-b12b-d7a592adde89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35f476aac39042d2b638c8ce68cb0866)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:43:29.532438 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db77e18d-f447-4c86-b12b-d7a592adde89 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:43:29.624027 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:43:29.631003 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-db77e18d-f447-4c86-b12b-d7a592adde89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35f476aac39042d2b638c8ce68cb0866'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:43:29.633357 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-db77e18d-f447-4c86-b12b-d7a592adde89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35f476aac39042d2b638c8ce68cb0866'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:43:29.651911 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-db77e18d-f447-4c86-b12b-d7a592adde89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35f476aac39042d2b638c8ce68cb0866'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:43:29.670173 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-db77e18d-f447-4c86-b12b-d7a592adde89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35f476aac39042d2b638c8ce68cb0866'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:43:29.671002 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-db77e18d-f447-4c86-b12b-d7a592adde89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35f476aac39042d2b638c8ce68cb0866'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:43:29.688400 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1178/2350] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:43:29 2022] DELETE /identity/v3/users/35f476aac39042d2b638c8ce68cb0866 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:43:30.039279 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b118b45-0ea5-488a-93d9-3408a87fdf41 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:43:30.041684 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-4b118b45-0ea5-488a-93d9-3408a87fdf41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:43:30.080757 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b118b45-0ea5-488a-93d9-3408a87fdf41 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:43:30.228085 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b225c4e3-41a6-4470-a844-28b73900889b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:43:30.229150 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b225c4e3-41a6-4470-a844-28b73900889b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:43:30.229731 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b225c4e3-41a6-4470-a844-28b73900889b admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:43:30.230409 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b225c4e3-41a6-4470-a844-28b73900889b admin admin] PATH_INFO: `/v3/projects/40395b3a79a444c9af480a4e8f5b7731` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:43:30.243720 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b225c4e3-41a6-4470-a844-28b73900889b admin admin] RBAC: Authorizing `identity:delete_project(project_id=40395b3a79a444c9af480a4e8f5b7731)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:43:30.254011 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b225c4e3-41a6-4470-a844-28b73900889b admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:43:30.330904 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:43:30.374327 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-b225c4e3-41a6-4470-a844-28b73900889b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40395b3a79a444c9af480a4e8f5b7731', 'request_id': 'req-b225c4e3-41a6-4470-a844-28b73900889b'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:43:30.393227 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-b225c4e3-41a6-4470-a844-28b73900889b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40395b3a79a444c9af480a4e8f5b7731', 'request_id': 'req-b225c4e3-41a6-4470-a844-28b73900889b'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:43:30.404292 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-b225c4e3-41a6-4470-a844-28b73900889b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40395b3a79a444c9af480a4e8f5b7731', 'request_id': 'req-b225c4e3-41a6-4470-a844-28b73900889b'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:43:30.427430 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-b225c4e3-41a6-4470-a844-28b73900889b admin admin] The token cache is being invalidate because project 40395b3a79a444c9af480a4e8f5b7731 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81618) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:43:30.427928 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-b225c4e3-41a6-4470-a844-28b73900889b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:43:30.429126 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-b225c4e3-41a6-4470-a844-28b73900889b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:43:30.432425 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1173/2351] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:43:30 2022] DELETE /identity/v3/projects/40395b3a79a444c9af480a4e8f5b7731 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:43:30.437811 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8188234-629b-47ed-b4bf-a64fd25a5a1f None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:43:30.438142 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8188234-629b-47ed-b4bf-a64fd25a5a1f None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:43:30.438362 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8188234-629b-47ed-b4bf-a64fd25a5a1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:43:30.461253 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-b8188234-629b-47ed-b4bf-a64fd25a5a1f None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:43:30.462336 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-b8188234-629b-47ed-b4bf-a64fd25a5a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:43:30.553138 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1179/2352] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:43:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:43:30.565264 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0e356c-4840-4b7f-904d-fdc085a7dfcf None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:43:30.566415 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0e356c-4840-4b7f-904d-fdc085a7dfcf None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:43:30.567308 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0e356c-4840-4b7f-904d-fdc085a7dfcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:43:30.616457 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-7b0e356c-4840-4b7f-904d-fdc085a7dfcf None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:43:30.619395 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-7b0e356c-4840-4b7f-904d-fdc085a7dfcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:43:30.630966 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1174/2353] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:43:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:43:30.636531 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8188234-629b-47ed-b4bf-a64fd25a5a1f None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:43:30.650831 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8188234-629b-47ed-b4bf-a64fd25a5a1f None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:43:30.661157 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d67087f-d74d-449a-b2dd-0faa29e8e4b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:43:30.665959 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d67087f-d74d-449a-b2dd-0faa29e8e4b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:43:30.666666 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d67087f-d74d-449a-b2dd-0faa29e8e4b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:43:30.667103 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d67087f-d74d-449a-b2dd-0faa29e8e4b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:43:30.668016 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d67087f-d74d-449a-b2dd-0faa29e8e4b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:43:30.668778 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d67087f-d74d-449a-b2dd-0faa29e8e4b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:43:30.670735 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d67087f-d74d-449a-b2dd-0faa29e8e4b7 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:43:30.693669 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1180/2354] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:43:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:43:30.699208 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0e356c-4840-4b7f-904d-fdc085a7dfcf None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:43:30.721908 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0e356c-4840-4b7f-904d-fdc085a7dfcf None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:43:30.732589 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09799179-4947-4d87-a8ae-4a6bf6d21a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:43:30.734709 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09799179-4947-4d87-a8ae-4a6bf6d21a83 admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:43:30.735450 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09799179-4947-4d87-a8ae-4a6bf6d21a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:43:30.736398 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09799179-4947-4d87-a8ae-4a6bf6d21a83 admin admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:43:30.739162 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09799179-4947-4d87-a8ae-4a6bf6d21a83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:43:30.743997 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09799179-4947-4d87-a8ae-4a6bf6d21a83 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:43:30.834949 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1175/2355] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:43:30 2022] POST /identity/v3/projects => generated 379 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:43:30.839047 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d67087f-d74d-449a-b2dd-0faa29e8e4b7 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:43:30.851411 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d67087f-d74d-449a-b2dd-0faa29e8e4b7 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:43:31.041706 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37675013-eca8-4591-81f7-15c9df2598d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:43:31.042595 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37675013-eca8-4591-81f7-15c9df2598d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:43:31.042871 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37675013-eca8-4591-81f7-15c9df2598d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:43:31.043142 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37675013-eca8-4591-81f7-15c9df2598d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:43:31.043783 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37675013-eca8-4591-81f7-15c9df2598d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:43:31.045690 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37675013-eca8-4591-81f7-15c9df2598d6 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:43:31.071018 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1181/2356] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:43:30 2022] POST /identity/v3/users => generated 290 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:43:31.075463 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09799179-4947-4d87-a8ae-4a6bf6d21a83 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:43:31.087523 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09799179-4947-4d87-a8ae-4a6bf6d21a83 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:43:31.092407 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561758fc-b7bd-48e1-bd7f-5d74f69fdd5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:43:31.093037 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561758fc-b7bd-48e1-bd7f-5d74f69fdd5f admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:43:31.093304 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561758fc-b7bd-48e1-bd7f-5d74f69fdd5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:43:31.094105 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561758fc-b7bd-48e1-bd7f-5d74f69fdd5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:43:31.094800 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561758fc-b7bd-48e1-bd7f-5d74f69fdd5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:43:31.096448 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561758fc-b7bd-48e1-bd7f-5d74f69fdd5f admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:43:31.111030 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1176/2357] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:43:31 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:43:31.116800 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37675013-eca8-4591-81f7-15c9df2598d6 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:43:31.135173 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37675013-eca8-4591-81f7-15c9df2598d6 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:43:31.141748 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83154ec8-ea21-4e87-bebc-2467c2d30034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:43:31.143057 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83154ec8-ea21-4e87-bebc-2467c2d30034 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:43:31.143057 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83154ec8-ea21-4e87-bebc-2467c2d30034 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:43:31.143057 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83154ec8-ea21-4e87-bebc-2467c2d30034 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:43:31.144119 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83154ec8-ea21-4e87-bebc-2467c2d30034 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:43:31.145997 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83154ec8-ea21-4e87-bebc-2467c2d30034 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:43:31.166127 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1182/2358] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:43:31 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:43:31.170410 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561758fc-b7bd-48e1-bd7f-5d74f69fdd5f admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:43:31.181996 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561758fc-b7bd-48e1-bd7f-5d74f69fdd5f admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:43:31.188568 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36470d8d-409c-45b5-81ed-75f399cb45c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:43:31.189645 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36470d8d-409c-45b5-81ed-75f399cb45c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:43:31.190028 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36470d8d-409c-45b5-81ed-75f399cb45c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:43:31.190436 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36470d8d-409c-45b5-81ed-75f399cb45c1 admin admin] PATH_INFO: `/v3/projects/f9eb08e4a6fc4434833ea75c5aee76ef/users/fbc7be7ad531431ebb63c6cd4add4722/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:43:31.208469 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36470d8d-409c-45b5-81ed-75f399cb45c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9eb08e4a6fc4434833ea75c5aee76ef, user_id=fbc7be7ad531431ebb63c6cd4add4722, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:43:31.211338 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36470d8d-409c-45b5-81ed-75f399cb45c1 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:43:31.242439 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1177/2359] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:43:31 2022] PUT /identity/v3/projects/f9eb08e4a6fc4434833ea75c5aee76ef/users/fbc7be7ad531431ebb63c6cd4add4722/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:43:31.247149 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beff369-e978-406e-9b7d-36aed5fa793b None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:43:31.247437 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beff369-e978-406e-9b7d-36aed5fa793b None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:43:31.247716 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beff369-e978-406e-9b7d-36aed5fa793b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:43:31.280590 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-4beff369-e978-406e-9b7d-36aed5fa793b None None] MFA Rules not processed for user `fbc7be7ad531431ebb63c6cd4add4722`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:43:31.282398 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-4beff369-e978-406e-9b7d-36aed5fa793b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:43:31.308460 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:43:31.394678 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1183/2360] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:43:31 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:43:31.402704 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36470d8d-409c-45b5-81ed-75f399cb45c1 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:43:31.481269 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36470d8d-409c-45b5-81ed-75f399cb45c1 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:43:31.487312 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45245e4f-ac8b-4916-8efe-75836c31f94d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:43:31.488419 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45245e4f-ac8b-4916-8efe-75836c31f94d admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:43:31.488718 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45245e4f-ac8b-4916-8efe-75836c31f94d admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:43:31.488968 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45245e4f-ac8b-4916-8efe-75836c31f94d admin admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:43:31.489759 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45245e4f-ac8b-4916-8efe-75836c31f94d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:43:31.491762 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45245e4f-ac8b-4916-8efe-75836c31f94d admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:43:31.528380 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1178/2361] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:43:31 2022] POST /identity/v3/projects => generated 381 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:43:31.534482 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beff369-e978-406e-9b7d-36aed5fa793b None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:43:31.547632 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beff369-e978-406e-9b7d-36aed5fa793b None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:43:31.638798 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb0f808-a5c1-41d9-b6ba-690fe2cb9fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:43:31.639487 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb0f808-a5c1-41d9-b6ba-690fe2cb9fd9 admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:43:31.640985 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb0f808-a5c1-41d9-b6ba-690fe2cb9fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:43:31.641275 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb0f808-a5c1-41d9-b6ba-690fe2cb9fd9 admin admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:43:31.641942 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb0f808-a5c1-41d9-b6ba-690fe2cb9fd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:43:31.645735 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb0f808-a5c1-41d9-b6ba-690fe2cb9fd9 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:43:31.687194 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1184/2362] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:43:31 2022] POST /identity/v3/users => generated 297 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:43:31.689748 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45245e4f-ac8b-4916-8efe-75836c31f94d admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:43:31.736408 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45245e4f-ac8b-4916-8efe-75836c31f94d admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:43:31.744426 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b284c41-353f-4e69-ada5-62095fe0ffe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:43:31.745345 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b284c41-353f-4e69-ada5-62095fe0ffe2 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:43:31.746262 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b284c41-353f-4e69-ada5-62095fe0ffe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:43:31.746602 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b284c41-353f-4e69-ada5-62095fe0ffe2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:43:31.747342 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b284c41-353f-4e69-ada5-62095fe0ffe2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:43:31.748892 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b284c41-353f-4e69-ada5-62095fe0ffe2 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:43:31.770282 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1179/2363] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:43:31 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:43:31.772866 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb0f808-a5c1-41d9-b6ba-690fe2cb9fd9 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:43:31.793947 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb0f808-a5c1-41d9-b6ba-690fe2cb9fd9 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:43:31.813123 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fe9b1f-8dda-41b7-853c-3e8a933a1d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:43:31.813819 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe9b1f-8dda-41b7-853c-3e8a933a1d64 admin admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:43:31.814102 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe9b1f-8dda-41b7-853c-3e8a933a1d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:43:31.814393 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe9b1f-8dda-41b7-853c-3e8a933a1d64 admin admin] PATH_INFO: `/v3/projects/4aac57fc4c2045bebf6c034fa8a9a88d/users/bd2cd2c026fe4db38113eccb54133cb1/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:43:31.839007 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fe9b1f-8dda-41b7-853c-3e8a933a1d64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4aac57fc4c2045bebf6c034fa8a9a88d, user_id=bd2cd2c026fe4db38113eccb54133cb1, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:43:31.841732 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fe9b1f-8dda-41b7-853c-3e8a933a1d64 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:43:31.867865 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1185/2364] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:43:31 2022] PUT /identity/v3/projects/4aac57fc4c2045bebf6c034fa8a9a88d/users/bd2cd2c026fe4db38113eccb54133cb1/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:43:31.876860 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b284c41-353f-4e69-ada5-62095fe0ffe2 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:43:31.892591 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b284c41-353f-4e69-ada5-62095fe0ffe2 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:43:31.997269 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:43:32.058646 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fe9b1f-8dda-41b7-853c-3e8a933a1d64 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:43:32.060748 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-d9fe9b1f-8dda-41b7-853c-3e8a933a1d64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:43:32.083036 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0222e1ec-3396-48c6-8039-4ff77e8ce7e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:43:32.084271 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0222e1ec-3396-48c6-8039-4ff77e8ce7e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:43:32.084756 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0222e1ec-3396-48c6-8039-4ff77e8ce7e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:43:32.085039 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0222e1ec-3396-48c6-8039-4ff77e8ce7e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:43:32.085933 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0222e1ec-3396-48c6-8039-4ff77e8ce7e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:43:32.087969 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0222e1ec-3396-48c6-8039-4ff77e8ce7e8 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:43:32.104450 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1180/2365] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:43:31 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:43:32.110182 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0222e1ec-3396-48c6-8039-4ff77e8ce7e8 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:43:32.123935 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0222e1ec-3396-48c6-8039-4ff77e8ce7e8 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:43:32.137251 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29da9d0a-7a9b-4f8b-b9e8-e0a70a2e8ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:43:32.138038 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29da9d0a-7a9b-4f8b-b9e8-e0a70a2e8ea6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:43:32.138330 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29da9d0a-7a9b-4f8b-b9e8-e0a70a2e8ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:43:32.138650 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29da9d0a-7a9b-4f8b-b9e8-e0a70a2e8ea6 admin admin] PATH_INFO: `/v3/projects/4aac57fc4c2045bebf6c034fa8a9a88d/users/bd2cd2c026fe4db38113eccb54133cb1/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:43:32.141504 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29da9d0a-7a9b-4f8b-b9e8-e0a70a2e8ea6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4aac57fc4c2045bebf6c034fa8a9a88d, user_id=bd2cd2c026fe4db38113eccb54133cb1, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:43:32.144594 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29da9d0a-7a9b-4f8b-b9e8-e0a70a2e8ea6 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:43:32.183723 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1181/2366] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:43:32 2022] PUT /identity/v3/projects/4aac57fc4c2045bebf6c034fa8a9a88d/users/bd2cd2c026fe4db38113eccb54133cb1/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:43:32.188044 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1214ad42-1e55-482d-abb8-76f7597b004e None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:43:32.189071 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1214ad42-1e55-482d-abb8-76f7597b004e None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:43:32.190241 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1214ad42-1e55-482d-abb8-76f7597b004e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:43:32.239916 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fe9b1f-8dda-41b7-853c-3e8a933a1d64 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:43:32.267491 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-1214ad42-1e55-482d-abb8-76f7597b004e None None] MFA Rules not processed for user `bd2cd2c026fe4db38113eccb54133cb1`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:43:32.270050 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-1214ad42-1e55-482d-abb8-76f7597b004e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:43:32.313197 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:43:32.406183 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1182/2367] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:43:32 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:43:32.458814 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b873d70-0def-42c6-9055-d03b8297cdb6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:43:32.460849 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b873d70-0def-42c6-9055-d03b8297cdb6 service nova] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:43:32.461072 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b873d70-0def-42c6-9055-d03b8297cdb6 service nova] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:43:32.461379 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b873d70-0def-42c6-9055-d03b8297cdb6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:43:32.464601 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-3b873d70-0def-42c6-9055-d03b8297cdb6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:43:32.491077 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b873d70-0def-42c6-9055-d03b8297cdb6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:43:32.494475 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b873d70-0def-42c6-9055-d03b8297cdb6 service nova] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:43:32.527948 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1186/2368] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:43:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:43:33.028511 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1214ad42-1e55-482d-abb8-76f7597b004e None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:43:33.030700 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-1214ad42-1e55-482d-abb8-76f7597b004e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:43:33.101581 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1214ad42-1e55-482d-abb8-76f7597b004e None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:43:33.106145 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336b3535-a259-4b2b-8913-c9eee421f294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:43:33.106825 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b3535-a259-4b2b-8913-c9eee421f294 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:43:33.107107 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b3535-a259-4b2b-8913-c9eee421f294 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:43:33.107383 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b3535-a259-4b2b-8913-c9eee421f294 admin admin] PATH_INFO: `/v3/users/9fd6c22a61be43658ae87052b867e384` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:43:33.123311 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336b3535-a259-4b2b-8913-c9eee421f294 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fd6c22a61be43658ae87052b867e384)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:43:33.125274 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336b3535-a259-4b2b-8913-c9eee421f294 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:43:33.156986 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:43:33.185665 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-336b3535-a259-4b2b-8913-c9eee421f294 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fd6c22a61be43658ae87052b867e384'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:43:33.194283 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-336b3535-a259-4b2b-8913-c9eee421f294 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fd6c22a61be43658ae87052b867e384'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:43:33.207234 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-336b3535-a259-4b2b-8913-c9eee421f294 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fd6c22a61be43658ae87052b867e384'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:43:33.216643 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-336b3535-a259-4b2b-8913-c9eee421f294 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fd6c22a61be43658ae87052b867e384'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:43:33.217677 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-336b3535-a259-4b2b-8913-c9eee421f294 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fd6c22a61be43658ae87052b867e384'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:43:33.228680 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1183/2369] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:43:33 2022] DELETE /identity/v3/users/9fd6c22a61be43658ae87052b867e384 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:43:33.419178 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b873d70-0def-42c6-9055-d03b8297cdb6 service nova] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:43:33.434862 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b873d70-0def-42c6-9055-d03b8297cdb6 service nova] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:43:33.505498 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c20d39-56c6-43f4-87e9-b75938b888a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:43:33.506140 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c20d39-56c6-43f4-87e9-b75938b888a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:43:33.506465 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c20d39-56c6-43f4-87e9-b75938b888a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:43:33.506741 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c20d39-56c6-43f4-87e9-b75938b888a2 admin admin] PATH_INFO: `/v3/projects/2a726afa4ec84ec99a5b1bd08c95a0cd` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:43:33.507910 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c20d39-56c6-43f4-87e9-b75938b888a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a726afa4ec84ec99a5b1bd08c95a0cd)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:43:33.509834 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c20d39-56c6-43f4-87e9-b75938b888a2 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:43:33.559760 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-47c20d39-56c6-43f4-87e9-b75938b888a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'request_id': 'req-47c20d39-56c6-43f4-87e9-b75938b888a2'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:43:33.566724 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-47c20d39-56c6-43f4-87e9-b75938b888a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'request_id': 'req-47c20d39-56c6-43f4-87e9-b75938b888a2'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:43:33.573048 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-47c20d39-56c6-43f4-87e9-b75938b888a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'request_id': 'req-47c20d39-56c6-43f4-87e9-b75938b888a2'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:43:33.590647 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-47c20d39-56c6-43f4-87e9-b75938b888a2 admin admin] The token cache is being invalidate because project 2a726afa4ec84ec99a5b1bd08c95a0cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81617) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:43:33.591129 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-47c20d39-56c6-43f4-87e9-b75938b888a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:43:33.591815 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-47c20d39-56c6-43f4-87e9-b75938b888a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:43:33.593323 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1187/2370] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:43:33 2022] DELETE /identity/v3/projects/2a726afa4ec84ec99a5b1bd08c95a0cd => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:43:33.597501 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336b3535-a259-4b2b-8913-c9eee421f294 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:43:33.599055 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-336b3535-a259-4b2b-8913-c9eee421f294 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:43:33.613002 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336b3535-a259-4b2b-8913-c9eee421f294 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:43:33.683243 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c8089e-984e-4765-b6f9-a6a6560230a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:43:33.686509 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c8089e-984e-4765-b6f9-a6a6560230a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:43:33.687759 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c8089e-984e-4765-b6f9-a6a6560230a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:43:33.688078 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c8089e-984e-4765-b6f9-a6a6560230a0 admin admin] PATH_INFO: `/v3/users/a8d61d084bf948679e9f849bbd743bc6` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:43:33.714593 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c8089e-984e-4765-b6f9-a6a6560230a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8d61d084bf948679e9f849bbd743bc6)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:43:33.716709 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c8089e-984e-4765-b6f9-a6a6560230a0 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:43:33.768919 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-85c8089e-984e-4765-b6f9-a6a6560230a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8d61d084bf948679e9f849bbd743bc6'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:43:33.776797 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-85c8089e-984e-4765-b6f9-a6a6560230a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8d61d084bf948679e9f849bbd743bc6'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:43:33.789923 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-85c8089e-984e-4765-b6f9-a6a6560230a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8d61d084bf948679e9f849bbd743bc6'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:43:33.798971 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-85c8089e-984e-4765-b6f9-a6a6560230a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8d61d084bf948679e9f849bbd743bc6'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:43:33.799484 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-85c8089e-984e-4765-b6f9-a6a6560230a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8d61d084bf948679e9f849bbd743bc6'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:43:33.806226 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1184/2371] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:43:33 2022] DELETE /identity/v3/users/a8d61d084bf948679e9f849bbd743bc6 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:43:33.988976 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c20d39-56c6-43f4-87e9-b75938b888a2 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:43:33.990526 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-47c20d39-56c6-43f4-87e9-b75938b888a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:43:34.004599 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c20d39-56c6-43f4-87e9-b75938b888a2 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:43:34.043809 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:43:34.075673 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95779e1-5422-4749-b6c5-40ab2cf2f6a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:43:34.076343 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95779e1-5422-4749-b6c5-40ab2cf2f6a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:43:34.076617 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95779e1-5422-4749-b6c5-40ab2cf2f6a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:43:34.076980 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95779e1-5422-4749-b6c5-40ab2cf2f6a1 admin admin] PATH_INFO: `/v3/projects/7b1b5a5be7314be08493869c523266bf` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:43:34.089701 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95779e1-5422-4749-b6c5-40ab2cf2f6a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b1b5a5be7314be08493869c523266bf)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:43:34.092219 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95779e1-5422-4749-b6c5-40ab2cf2f6a1 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:43:34.151892 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-e95779e1-5422-4749-b6c5-40ab2cf2f6a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b1b5a5be7314be08493869c523266bf', 'request_id': 'req-e95779e1-5422-4749-b6c5-40ab2cf2f6a1'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:43:34.160251 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-e95779e1-5422-4749-b6c5-40ab2cf2f6a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b1b5a5be7314be08493869c523266bf', 'request_id': 'req-e95779e1-5422-4749-b6c5-40ab2cf2f6a1'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:43:34.166468 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-e95779e1-5422-4749-b6c5-40ab2cf2f6a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b1b5a5be7314be08493869c523266bf', 'request_id': 'req-e95779e1-5422-4749-b6c5-40ab2cf2f6a1'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:43:34.181346 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-e95779e1-5422-4749-b6c5-40ab2cf2f6a1 admin admin] The token cache is being invalidate because project 7b1b5a5be7314be08493869c523266bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81617) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:43:34.181629 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-e95779e1-5422-4749-b6c5-40ab2cf2f6a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:43:34.182262 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-e95779e1-5422-4749-b6c5-40ab2cf2f6a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:43:34.183933 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1188/2372] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:43:33 2022] DELETE /identity/v3/projects/7b1b5a5be7314be08493869c523266bf => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:43:38.178416 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c8089e-984e-4765-b6f9-a6a6560230a0 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:43:38.180361 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-85c8089e-984e-4765-b6f9-a6a6560230a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:43:38.196909 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:43:38.267805 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c8089e-984e-4765-b6f9-a6a6560230a0 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:43:38.333933 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4515096-9844-4768-91c6-2f708293c41b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:43:38.334583 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4515096-9844-4768-91c6-2f708293c41b service nova] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:43:38.334864 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4515096-9844-4768-91c6-2f708293c41b service nova] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:43:38.335142 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4515096-9844-4768-91c6-2f708293c41b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:43:38.336968 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-c4515096-9844-4768-91c6-2f708293c41b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:43:38.349447 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4515096-9844-4768-91c6-2f708293c41b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:43:38.351075 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4515096-9844-4768-91c6-2f708293c41b service nova] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:43:38.362522 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1185/2373] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:43:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:43:41.652411 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95779e1-5422-4749-b6c5-40ab2cf2f6a1 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:43:41.654032 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-e95779e1-5422-4749-b6c5-40ab2cf2f6a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:43:41.670322 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95779e1-5422-4749-b6c5-40ab2cf2f6a1 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:43:41.676497 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0bb133-f301-4481-a0cf-55716a5d80d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:43:41.677200 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0bb133-f301-4481-a0cf-55716a5d80d7 service nova] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:43:41.677543 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0bb133-f301-4481-a0cf-55716a5d80d7 service nova] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:43:41.677882 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0bb133-f301-4481-a0cf-55716a5d80d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:43:41.679893 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-bc0bb133-f301-4481-a0cf-55716a5d80d7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:43:41.692359 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0bb133-f301-4481-a0cf-55716a5d80d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:43:41.693940 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0bb133-f301-4481-a0cf-55716a5d80d7 service nova] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:43:41.704506 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1189/2374] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:43:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:43:48.357239 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=56 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:43:51.700074 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=45,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:44:32.472037 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4515096-9844-4768-91c6-2f708293c41b service nova] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:44:32.473625 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-c4515096-9844-4768-91c6-2f708293c41b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:44:32.540907 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4515096-9844-4768-91c6-2f708293c41b service nova] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:44:32.549787 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caebe3b6-b467-4fbb-bf49-920a51ef321f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:44:32.551197 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caebe3b6-b467-4fbb-bf49-920a51ef321f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:44:32.551493 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caebe3b6-b467-4fbb-bf49-920a51ef321f admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:44:32.551770 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caebe3b6-b467-4fbb-bf49-920a51ef321f admin admin] PATH_INFO: `/v3/users/fbc7be7ad531431ebb63c6cd4add4722` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:44:32.555359 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caebe3b6-b467-4fbb-bf49-920a51ef321f admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbc7be7ad531431ebb63c6cd4add4722)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:44:32.559595 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caebe3b6-b467-4fbb-bf49-920a51ef321f admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:44:32.608837 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-caebe3b6-b467-4fbb-bf49-920a51ef321f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbc7be7ad531431ebb63c6cd4add4722'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:44:32.615794 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-caebe3b6-b467-4fbb-bf49-920a51ef321f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbc7be7ad531431ebb63c6cd4add4722'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:44:32.628096 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-caebe3b6-b467-4fbb-bf49-920a51ef321f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbc7be7ad531431ebb63c6cd4add4722'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:44:32.635458 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-caebe3b6-b467-4fbb-bf49-920a51ef321f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbc7be7ad531431ebb63c6cd4add4722'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:44:32.636132 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-caebe3b6-b467-4fbb-bf49-920a51ef321f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbc7be7ad531431ebb63c6cd4add4722'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:44:32.643052 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1186/2375] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:44:32 2022] DELETE /identity/v3/users/fbc7be7ad531431ebb63c6cd4add4722 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:44:32.647655 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe823f-d056-4610-a303-e9921e88ed0b None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:44:32.648158 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe823f-d056-4610-a303-e9921e88ed0b None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:44:32.648268 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe823f-d056-4610-a303-e9921e88ed0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:44:32.666682 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-9ffe823f-d056-4610-a303-e9921e88ed0b None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:44:32.667877 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-9ffe823f-d056-4610-a303-e9921e88ed0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:44:32.726221 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1190/2376] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:44:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:44:32.736013 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caebe3b6-b467-4fbb-bf49-920a51ef321f admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:44:32.737436 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-caebe3b6-b467-4fbb-bf49-920a51ef321f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:44:32.796212 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caebe3b6-b467-4fbb-bf49-920a51ef321f admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:44:32.799864 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e148e1c-25ae-40c1-8adc-b29305402180 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:44:32.800464 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e148e1c-25ae-40c1-8adc-b29305402180 service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:44:32.800745 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e148e1c-25ae-40c1-8adc-b29305402180 service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:44:32.801017 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e148e1c-25ae-40c1-8adc-b29305402180 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:44:32.810047 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e148e1c-25ae-40c1-8adc-b29305402180 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:44:32.811985 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e148e1c-25ae-40c1-8adc-b29305402180 service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:44:32.821291 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1187/2377] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:44:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:44:33.096030 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe823f-d056-4610-a303-e9921e88ed0b None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:44:33.097996 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-9ffe823f-d056-4610-a303-e9921e88ed0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:44:33.114155 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe823f-d056-4610-a303-e9921e88ed0b None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:44:33.120107 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d73b33b-c50c-4d51-a7bb-8ea825898946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:44:33.120740 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d73b33b-c50c-4d51-a7bb-8ea825898946 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:44:33.121015 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d73b33b-c50c-4d51-a7bb-8ea825898946 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:44:33.121291 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d73b33b-c50c-4d51-a7bb-8ea825898946 admin admin] PATH_INFO: `/v3/projects/f9eb08e4a6fc4434833ea75c5aee76ef` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:44:33.123218 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d73b33b-c50c-4d51-a7bb-8ea825898946 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9eb08e4a6fc4434833ea75c5aee76ef)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:44:33.125798 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d73b33b-c50c-4d51-a7bb-8ea825898946 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:44:33.179186 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-8d73b33b-c50c-4d51-a7bb-8ea825898946 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9eb08e4a6fc4434833ea75c5aee76ef', 'request_id': 'req-8d73b33b-c50c-4d51-a7bb-8ea825898946'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:44:33.185589 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-8d73b33b-c50c-4d51-a7bb-8ea825898946 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9eb08e4a6fc4434833ea75c5aee76ef', 'request_id': 'req-8d73b33b-c50c-4d51-a7bb-8ea825898946'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:44:33.191891 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-8d73b33b-c50c-4d51-a7bb-8ea825898946 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9eb08e4a6fc4434833ea75c5aee76ef', 'request_id': 'req-8d73b33b-c50c-4d51-a7bb-8ea825898946'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:44:33.207780 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-8d73b33b-c50c-4d51-a7bb-8ea825898946 admin admin] The token cache is being invalidate because project f9eb08e4a6fc4434833ea75c5aee76ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81617) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:44:33.208084 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-8d73b33b-c50c-4d51-a7bb-8ea825898946 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:44:33.208739 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-8d73b33b-c50c-4d51-a7bb-8ea825898946 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:44:33.210285 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1191/2378] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:44:33 2022] DELETE /identity/v3/projects/f9eb08e4a6fc4434833ea75c5aee76ef => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:44:33.214564 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e148e1c-25ae-40c1-8adc-b29305402180 service neutron] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:44:33.224787 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e148e1c-25ae-40c1-8adc-b29305402180 service neutron] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:44:33.244016 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:44:33.284831 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d1ffde-5359-44ee-9684-1de91933f699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:44:33.286109 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d1ffde-5359-44ee-9684-1de91933f699 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:44:33.286424 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d1ffde-5359-44ee-9684-1de91933f699 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:44:33.286705 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d1ffde-5359-44ee-9684-1de91933f699 admin admin] PATH_INFO: `/v3/users/bd2cd2c026fe4db38113eccb54133cb1` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:44:33.287884 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d1ffde-5359-44ee-9684-1de91933f699 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd2cd2c026fe4db38113eccb54133cb1)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:44:33.289799 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d1ffde-5359-44ee-9684-1de91933f699 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:44:33.338950 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-f6d1ffde-5359-44ee-9684-1de91933f699 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd2cd2c026fe4db38113eccb54133cb1'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:44:33.347328 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-f6d1ffde-5359-44ee-9684-1de91933f699 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd2cd2c026fe4db38113eccb54133cb1'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:44:33.360188 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-f6d1ffde-5359-44ee-9684-1de91933f699 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd2cd2c026fe4db38113eccb54133cb1'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:44:33.368113 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-f6d1ffde-5359-44ee-9684-1de91933f699 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd2cd2c026fe4db38113eccb54133cb1'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:44:33.368745 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-f6d1ffde-5359-44ee-9684-1de91933f699 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd2cd2c026fe4db38113eccb54133cb1'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:44:33.375359 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1188/2379] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:44:33 2022] DELETE /identity/v3/users/bd2cd2c026fe4db38113eccb54133cb1 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:44:33.658290 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d73b33b-c50c-4d51-a7bb-8ea825898946 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:44:33.660835 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-8d73b33b-c50c-4d51-a7bb-8ea825898946 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:44:33.679402 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d73b33b-c50c-4d51-a7bb-8ea825898946 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:44:33.741918 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a340ea6-dfcf-418c-b008-8189735b1b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:44:33.742585 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a340ea6-dfcf-418c-b008-8189735b1b61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:44:33.742858 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a340ea6-dfcf-418c-b008-8189735b1b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:44:33.743130 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a340ea6-dfcf-418c-b008-8189735b1b61 admin admin] PATH_INFO: `/v3/projects/4aac57fc4c2045bebf6c034fa8a9a88d` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:44:33.744251 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a340ea6-dfcf-418c-b008-8189735b1b61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4aac57fc4c2045bebf6c034fa8a9a88d)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:44:33.746129 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a340ea6-dfcf-418c-b008-8189735b1b61 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:44:33.777436 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:44:33.795554 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-3a340ea6-dfcf-418c-b008-8189735b1b61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4aac57fc4c2045bebf6c034fa8a9a88d', 'request_id': 'req-3a340ea6-dfcf-418c-b008-8189735b1b61'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:44:33.801949 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-3a340ea6-dfcf-418c-b008-8189735b1b61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4aac57fc4c2045bebf6c034fa8a9a88d', 'request_id': 'req-3a340ea6-dfcf-418c-b008-8189735b1b61'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:44:33.807999 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-3a340ea6-dfcf-418c-b008-8189735b1b61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4aac57fc4c2045bebf6c034fa8a9a88d', 'request_id': 'req-3a340ea6-dfcf-418c-b008-8189735b1b61'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:44:33.820461 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-3a340ea6-dfcf-418c-b008-8189735b1b61 admin admin] The token cache is being invalidate because project 4aac57fc4c2045bebf6c034fa8a9a88d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81617) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:44:33.820751 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-3a340ea6-dfcf-418c-b008-8189735b1b61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:44:33.821386 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-3a340ea6-dfcf-418c-b008-8189735b1b61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:44:33.822899 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1192/2380] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:44:33 2022] DELETE /identity/v3/projects/4aac57fc4c2045bebf6c034fa8a9a88d => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:44:33.828587 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e2a8f1-afb1-47a2-9523-c280367e02c3 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:44:33.829074 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e2a8f1-afb1-47a2-9523-c280367e02c3 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:44:33.829192 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e2a8f1-afb1-47a2-9523-c280367e02c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:44:33.849517 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-53e2a8f1-afb1-47a2-9523-c280367e02c3 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:44:33.850777 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-53e2a8f1-afb1-47a2-9523-c280367e02c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:44:33.911568 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1189/2381] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:44:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:44:33.919629 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da9e85c-a96d-478f-b883-909965845072 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:44:33.919915 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da9e85c-a96d-478f-b883-909965845072 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:44:33.920202 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da9e85c-a96d-478f-b883-909965845072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:44:33.940567 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-8da9e85c-a96d-478f-b883-909965845072 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:44:33.941783 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-8da9e85c-a96d-478f-b883-909965845072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:44:33.949889 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1193/2382] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:44:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:44:33.954734 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e2a8f1-afb1-47a2-9523-c280367e02c3 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:44:33.964681 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e2a8f1-afb1-47a2-9523-c280367e02c3 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:44:33.970059 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff54f577-48fd-45b4-b6f6-a35f41b0bc2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:44:33.970690 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff54f577-48fd-45b4-b6f6-a35f41b0bc2f admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:44:33.970971 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff54f577-48fd-45b4-b6f6-a35f41b0bc2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:44:33.971242 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff54f577-48fd-45b4-b6f6-a35f41b0bc2f admin admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:44:33.971891 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff54f577-48fd-45b4-b6f6-a35f41b0bc2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:44:33.972195 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff54f577-48fd-45b4-b6f6-a35f41b0bc2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:44:33.973783 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff54f577-48fd-45b4-b6f6-a35f41b0bc2f admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:44:33.985457 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1190/2383] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:44:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:44:33.990004 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da9e85c-a96d-478f-b883-909965845072 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:44:33.999463 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da9e85c-a96d-478f-b883-909965845072 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:44:34.004130 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71077202-b661-4f65-b6a0-50bb2ab12c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:44:34.004720 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71077202-b661-4f65-b6a0-50bb2ab12c99 admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:44:34.005002 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71077202-b661-4f65-b6a0-50bb2ab12c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:44:34.005282 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71077202-b661-4f65-b6a0-50bb2ab12c99 admin admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:44:34.005940 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71077202-b661-4f65-b6a0-50bb2ab12c99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:44:34.007904 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71077202-b661-4f65-b6a0-50bb2ab12c99 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:44:34.036667 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1194/2384] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:44:33 2022] POST /identity/v3/projects => generated 369 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 16:44:34.040583 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff54f577-48fd-45b4-b6f6-a35f41b0bc2f admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:44:34.051101 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff54f577-48fd-45b4-b6f6-a35f41b0bc2f admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:44:34.079889 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:44:34.112265 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e9c1bd-77d7-4a57-8bee-2d0d9e6eed29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:44:34.112877 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e9c1bd-77d7-4a57-8bee-2d0d9e6eed29 admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:44:34.113159 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e9c1bd-77d7-4a57-8bee-2d0d9e6eed29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:44:34.113431 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e9c1bd-77d7-4a57-8bee-2d0d9e6eed29 admin admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:44:34.114061 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e9c1bd-77d7-4a57-8bee-2d0d9e6eed29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:44:34.116286 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e9c1bd-77d7-4a57-8bee-2d0d9e6eed29 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:44:34.138032 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1191/2385] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:44:34 2022] POST /identity/v3/users => generated 285 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 16:44:34.143193 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71077202-b661-4f65-b6a0-50bb2ab12c99 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:44:34.155600 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71077202-b661-4f65-b6a0-50bb2ab12c99 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:44:34.160173 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127da7e1-e9f0-42f8-b865-652fc5a6748d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:44:34.160768 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127da7e1-e9f0-42f8-b865-652fc5a6748d admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:44:34.161044 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127da7e1-e9f0-42f8-b865-652fc5a6748d admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:44:34.161315 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127da7e1-e9f0-42f8-b865-652fc5a6748d admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:44:34.161948 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127da7e1-e9f0-42f8-b865-652fc5a6748d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:44:34.163558 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127da7e1-e9f0-42f8-b865-652fc5a6748d admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:44:34.173387 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1195/2386] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:44:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:44:34.177392 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e9c1bd-77d7-4a57-8bee-2d0d9e6eed29 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:44:34.188134 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e9c1bd-77d7-4a57-8bee-2d0d9e6eed29 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:44:34.192822 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ae662d-0315-4029-87be-f8d3db05e6ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:44:34.193410 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae662d-0315-4029-87be-f8d3db05e6ea admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:44:34.193691 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae662d-0315-4029-87be-f8d3db05e6ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:44:34.193963 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae662d-0315-4029-87be-f8d3db05e6ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:44:34.194624 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ae662d-0315-4029-87be-f8d3db05e6ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:44:34.196207 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ae662d-0315-4029-87be-f8d3db05e6ea admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:44:34.207603 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1192/2387] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:44:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:44:34.211550 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127da7e1-e9f0-42f8-b865-652fc5a6748d admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:44:34.221578 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127da7e1-e9f0-42f8-b865-652fc5a6748d admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:44:34.226180 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9688b9ae-ca49-489c-ad2c-56982fd560e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:44:34.226850 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9688b9ae-ca49-489c-ad2c-56982fd560e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:44:34.227133 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9688b9ae-ca49-489c-ad2c-56982fd560e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:44:34.227410 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9688b9ae-ca49-489c-ad2c-56982fd560e6 admin admin] PATH_INFO: `/v3/projects/039e29833028467786789a052a0b963e/users/765a46332add46a08dbcf383e1344549/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:44:34.241784 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9688b9ae-ca49-489c-ad2c-56982fd560e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=039e29833028467786789a052a0b963e, user_id=765a46332add46a08dbcf383e1344549, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:44:34.244584 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9688b9ae-ca49-489c-ad2c-56982fd560e6 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:44:34.263501 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1196/2388] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:44:34 2022] PUT /identity/v3/projects/039e29833028467786789a052a0b963e/users/765a46332add46a08dbcf383e1344549/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:44:34.275160 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c1a43d-b942-4b60-a202-ed3a436debea None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:44:34.275444 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c1a43d-b942-4b60-a202-ed3a436debea None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:44:34.276145 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c1a43d-b942-4b60-a202-ed3a436debea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:44:34.303934 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-86c1a43d-b942-4b60-a202-ed3a436debea None None] MFA Rules not processed for user `765a46332add46a08dbcf383e1344549`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:44:34.305331 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-86c1a43d-b942-4b60-a202-ed3a436debea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:44:34.371027 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1193/2389] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:44:34 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:44:34.376192 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9688b9ae-ca49-489c-ad2c-56982fd560e6 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:44:34.438505 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9688b9ae-ca49-489c-ad2c-56982fd560e6 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:44:34.443393 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c374f4b9-68d0-4bb8-af26-7a0a795f5df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:44:34.443964 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c374f4b9-68d0-4bb8-af26-7a0a795f5df2 admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:44:34.444238 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c374f4b9-68d0-4bb8-af26-7a0a795f5df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:44:34.444507 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c374f4b9-68d0-4bb8-af26-7a0a795f5df2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:44:34.445144 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c374f4b9-68d0-4bb8-af26-7a0a795f5df2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:44:34.447052 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c374f4b9-68d0-4bb8-af26-7a0a795f5df2 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:44:34.473725 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1197/2390] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:44:34 2022] POST /identity/v3/projects => generated 369 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:44:34.478113 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c1a43d-b942-4b60-a202-ed3a436debea None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:44:34.490459 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c1a43d-b942-4b60-a202-ed3a436debea None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:44:34.547975 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79843571-2d9f-4b48-a721-72764eb9636b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:44:34.548590 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79843571-2d9f-4b48-a721-72764eb9636b admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:44:34.548872 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79843571-2d9f-4b48-a721-72764eb9636b admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:44:34.549143 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79843571-2d9f-4b48-a721-72764eb9636b admin admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:44:34.549792 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79843571-2d9f-4b48-a721-72764eb9636b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:44:34.551733 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79843571-2d9f-4b48-a721-72764eb9636b admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:44:34.574208 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1194/2391] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:44:34 2022] POST /identity/v3/users => generated 291 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:44:34.577601 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c374f4b9-68d0-4bb8-af26-7a0a795f5df2 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:44:34.588124 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c374f4b9-68d0-4bb8-af26-7a0a795f5df2 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:44:34.592771 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8896b759-5063-48a5-aa1d-1b878b603e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:44:34.593384 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8896b759-5063-48a5-aa1d-1b878b603e26 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:44:34.593666 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8896b759-5063-48a5-aa1d-1b878b603e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:44:34.593942 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8896b759-5063-48a5-aa1d-1b878b603e26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:44:34.594631 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8896b759-5063-48a5-aa1d-1b878b603e26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:44:34.596226 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8896b759-5063-48a5-aa1d-1b878b603e26 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:44:34.607277 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1198/2392] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:44:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:44:34.612092 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79843571-2d9f-4b48-a721-72764eb9636b admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:44:34.629323 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79843571-2d9f-4b48-a721-72764eb9636b admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:44:34.634015 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ae53f9-3743-47fb-9859-bfd943529cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:44:34.634701 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae53f9-3743-47fb-9859-bfd943529cb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:44:34.634976 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae53f9-3743-47fb-9859-bfd943529cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:44:34.635248 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae53f9-3743-47fb-9859-bfd943529cb2 admin admin] PATH_INFO: `/v3/projects/9d56de62d4c14fbaaa1c6ab97eb41e4f/users/a60ebfbddb884708a303f1b9a439cbb9/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:44:34.650612 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ae53f9-3743-47fb-9859-bfd943529cb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d56de62d4c14fbaaa1c6ab97eb41e4f, user_id=a60ebfbddb884708a303f1b9a439cbb9, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:44:34.653252 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ae53f9-3743-47fb-9859-bfd943529cb2 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:44:34.666556 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1195/2393] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:44:34 2022] PUT /identity/v3/projects/9d56de62d4c14fbaaa1c6ab97eb41e4f/users/a60ebfbddb884708a303f1b9a439cbb9/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:44:34.671073 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8896b759-5063-48a5-aa1d-1b878b603e26 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:44:34.682105 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8896b759-5063-48a5-aa1d-1b878b603e26 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:44:34.712260 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:44:34.752760 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f0b5fd-22c2-434d-887d-846328bedeba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:44:34.753363 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f0b5fd-22c2-434d-887d-846328bedeba admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:44:34.753669 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f0b5fd-22c2-434d-887d-846328bedeba admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:44:34.753937 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f0b5fd-22c2-434d-887d-846328bedeba admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:44:34.754621 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f0b5fd-22c2-434d-887d-846328bedeba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:44:34.756279 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f0b5fd-22c2-434d-887d-846328bedeba admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:44:34.765992 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1199/2394] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:44:34 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:44:34.770166 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ae53f9-3743-47fb-9859-bfd943529cb2 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:44:34.775786 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:44:34.782494 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ae53f9-3743-47fb-9859-bfd943529cb2 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:44:34.789271 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366c8e58-317c-43fe-8108-fd9cf1d3f184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:44:34.789929 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c8e58-317c-43fe-8108-fd9cf1d3f184 admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:44:34.790210 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c8e58-317c-43fe-8108-fd9cf1d3f184 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:44:34.790546 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c8e58-317c-43fe-8108-fd9cf1d3f184 admin admin] PATH_INFO: `/v3/projects/9d56de62d4c14fbaaa1c6ab97eb41e4f/users/a60ebfbddb884708a303f1b9a439cbb9/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:44:34.792448 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366c8e58-317c-43fe-8108-fd9cf1d3f184 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d56de62d4c14fbaaa1c6ab97eb41e4f, user_id=a60ebfbddb884708a303f1b9a439cbb9, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:44:34.795673 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366c8e58-317c-43fe-8108-fd9cf1d3f184 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:44:34.806274 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1196/2395] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:44:34 2022] PUT /identity/v3/projects/9d56de62d4c14fbaaa1c6ab97eb41e4f/users/a60ebfbddb884708a303f1b9a439cbb9/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 16:44:34.817397 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b99e1-6372-4cdd-9d4d-fbe8ffe3cc8c None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:44:34.817676 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b99e1-6372-4cdd-9d4d-fbe8ffe3cc8c None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:44:34.817948 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b99e1-6372-4cdd-9d4d-fbe8ffe3cc8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:44:34.845402 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-f83b99e1-6372-4cdd-9d4d-fbe8ffe3cc8c None None] MFA Rules not processed for user `a60ebfbddb884708a303f1b9a439cbb9`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:44:34.846554 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-f83b99e1-6372-4cdd-9d4d-fbe8ffe3cc8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:44:34.914220 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1200/2396] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:44:34 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:44:44.802310 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=4,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:44:44.911233 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=52 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:45:04.427038 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366c8e58-317c-43fe-8108-fd9cf1d3f184 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:45:04.428847 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-366c8e58-317c-43fe-8108-fd9cf1d3f184 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:45:04.495813 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366c8e58-317c-43fe-8108-fd9cf1d3f184 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:45:04.499466 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfed5cf1-5563-4834-a6b0-61095d18e236 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:45:04.500068 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfed5cf1-5563-4834-a6b0-61095d18e236 service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:45:04.500341 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfed5cf1-5563-4834-a6b0-61095d18e236 service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:45:04.500606 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfed5cf1-5563-4834-a6b0-61095d18e236 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:45:04.502887 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-dfed5cf1-5563-4834-a6b0-61095d18e236 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:45:04.512173 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfed5cf1-5563-4834-a6b0-61095d18e236 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:45:04.513510 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfed5cf1-5563-4834-a6b0-61095d18e236 service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:45:04.521349 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1197/2397] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:45:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:45:12.942966 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83b99e1-6372-4cdd-9d4d-fbe8ffe3cc8c None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:45:12.944358 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-f83b99e1-6372-4cdd-9d4d-fbe8ffe3cc8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:45:12.999409 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83b99e1-6372-4cdd-9d4d-fbe8ffe3cc8c None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:45:13.003569 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dfc7a9-496c-46f1-872f-c2850a128d1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:45:13.004174 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dfc7a9-496c-46f1-872f-c2850a128d1a service cinder] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:45:13.004450 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dfc7a9-496c-46f1-872f-c2850a128d1a service cinder] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:45:13.004741 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dfc7a9-496c-46f1-872f-c2850a128d1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:45:13.006513 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-62dfc7a9-496c-46f1-872f-c2850a128d1a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:45:13.079729 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dfc7a9-496c-46f1-872f-c2850a128d1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:45:13.081075 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dfc7a9-496c-46f1-872f-c2850a128d1a service cinder] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:45:13.090701 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1201/2398] 10.222.0.57 () {60 vars in 1396 bytes} [Mon Sep 19 16:45:12 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3260 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:45:13.294727 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfed5cf1-5563-4834-a6b0-61095d18e236 service neutron] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:45:13.304707 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfed5cf1-5563-4834-a6b0-61095d18e236 service neutron] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:45:13.308619 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a0e137-d627-4105-9eba-6a731e85180f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:45:13.309229 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a0e137-d627-4105-9eba-6a731e85180f service cinder] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:45:13.309509 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a0e137-d627-4105-9eba-6a731e85180f service cinder] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:45:13.309785 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a0e137-d627-4105-9eba-6a731e85180f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:45:13.311602 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-38a0e137-d627-4105-9eba-6a731e85180f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:45:13.324618 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application [None req-38a0e137-d627-4105-9eba-6a731e85180f service cinder] Could not find project: 4aac57fc4c2045bebf6c034fa8a9a88d.: keystone.exception.ProjectNotFound: Could not find project: 4aac57fc4c2045bebf6c034fa8a9a88d. Sep 19 16:45:13.324618 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 16:45:13.324618 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 16:45:13.324618 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 16:45:13.324618 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 16:45:13.324618 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 16:45:13.324618 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 16:45:13.324618 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 16:45:13.324618 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 16:45:13.324618 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 16:45:13.324618 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 16:45:13.324618 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 16:45:13.324618 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Sep 19 16:45:13.324618 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Sep 19 16:45:13.324618 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Sep 19 16:45:13.327058 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Sep 19 16:45:13.327058 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Sep 19 16:45:13.327058 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Sep 19 16:45:13.327058 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:45:13.327058 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:45:13.327058 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Sep 19 16:45:13.327058 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Sep 19 16:45:13.327058 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:45:13.327058 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:45:13.327058 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:45:13.327058 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:45:13.327058 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:45:13.327058 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:45:13.327058 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:45:13.327058 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:45:13.327058 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:45:13.327058 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:45:13.330002 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:45:13.330002 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:45:13.330002 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:45:13.330002 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:45:13.330002 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Sep 19 16:45:13.330002 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Sep 19 16:45:13.330002 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Sep 19 16:45:13.330002 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application self._validate_token_resources() Sep 19 16:45:13.330002 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Sep 19 16:45:13.330002 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Sep 19 16:45:13.330002 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Sep 19 16:45:13.330002 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Sep 19 16:45:13.330002 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 16:45:13.330002 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 16:45:13.330002 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 16:45:13.330002 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 16:45:13.330002 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 16:45:13.332974 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 16:45:13.332974 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 16:45:13.332974 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application with Lock( Sep 19 16:45:13.332974 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 16:45:13.332974 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._enter() Sep 19 16:45:13.332974 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 16:45:13.332974 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 16:45:13.332974 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 16:45:13.332974 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.creator() Sep 19 16:45:13.332974 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 16:45:13.332974 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application created_value = creator( Sep 19 16:45:13.332974 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 16:45:13.332974 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 16:45:13.332974 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 16:45:13.332974 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 16:45:13.332974 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 16:45:13.332974 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 16:45:13.335273 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 4aac57fc4c2045bebf6c034fa8a9a88d. Sep 19 16:45:13.335273 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: ERROR keystone.server.flask.application Sep 19 16:45:13.335273 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1198/2399] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:45:13 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 32 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 19 16:45:13.696812 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dfc7a9-496c-46f1-872f-c2850a128d1a service cinder] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:45:13.705911 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dfc7a9-496c-46f1-872f-c2850a128d1a service cinder] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:45:13.709964 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6e4b06-3cfb-4408-af93-372290b61f69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:45:13.710617 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e4b06-3cfb-4408-af93-372290b61f69 service cinder] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:45:13.710898 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e4b06-3cfb-4408-af93-372290b61f69 service cinder] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:45:13.711170 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e4b06-3cfb-4408-af93-372290b61f69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:45:13.721392 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6e4b06-3cfb-4408-af93-372290b61f69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:45:13.722875 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6e4b06-3cfb-4408-af93-372290b61f69 service cinder] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:45:13.732869 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1202/2400] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:45:13 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:45:19.647493 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a0e137-d627-4105-9eba-6a731e85180f service cinder] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:45:19.648946 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-38a0e137-d627-4105-9eba-6a731e85180f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:45:19.661424 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a0e137-d627-4105-9eba-6a731e85180f service cinder] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:45:19.674708 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de553717-e833-40f5-8d64-da7b5a1f8255 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:45:19.674708 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de553717-e833-40f5-8d64-da7b5a1f8255 service cinder] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:45:19.674708 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de553717-e833-40f5-8d64-da7b5a1f8255 service cinder] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:45:19.674708 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de553717-e833-40f5-8d64-da7b5a1f8255 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:45:19.726160 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de553717-e833-40f5-8d64-da7b5a1f8255 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:45:19.727684 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de553717-e833-40f5-8d64-da7b5a1f8255 service cinder] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:45:19.737434 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1199/2401] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:45:19 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:45:23.728280 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=59 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:45:29.732711 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=62 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:46:13.260563 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6e4b06-3cfb-4408-af93-372290b61f69 service cinder] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:46:13.261987 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-8d6e4b06-3cfb-4408-af93-372290b61f69 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:46:13.349306 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6e4b06-3cfb-4408-af93-372290b61f69 service cinder] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:46:13.354832 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e170cbcc-d6bb-4db1-88f9-bfc2241c7620 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:46:13.355431 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e170cbcc-d6bb-4db1-88f9-bfc2241c7620 service nova] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:46:13.355708 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e170cbcc-d6bb-4db1-88f9-bfc2241c7620 service nova] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:46:13.355978 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e170cbcc-d6bb-4db1-88f9-bfc2241c7620 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:46:13.357670 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-e170cbcc-d6bb-4db1-88f9-bfc2241c7620 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:46:13.425767 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e170cbcc-d6bb-4db1-88f9-bfc2241c7620 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:46:13.427373 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e170cbcc-d6bb-4db1-88f9-bfc2241c7620 service nova] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:46:13.436692 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1203/2402] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:46:13 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:46:13.488726 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de553717-e833-40f5-8d64-da7b5a1f8255 service cinder] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:46:13.498554 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de553717-e833-40f5-8d64-da7b5a1f8255 service cinder] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:46:13.502297 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a972bc-9539-4f7f-ac26-12db42fd1bb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:46:13.503006 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a972bc-9539-4f7f-ac26-12db42fd1bb4 service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:46:13.503287 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a972bc-9539-4f7f-ac26-12db42fd1bb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:46:13.503559 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a972bc-9539-4f7f-ac26-12db42fd1bb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:46:13.505313 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-73a972bc-9539-4f7f-ac26-12db42fd1bb4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:46:13.520689 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a972bc-9539-4f7f-ac26-12db42fd1bb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:46:13.522758 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a972bc-9539-4f7f-ac26-12db42fd1bb4 service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:46:13.540164 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1200/2403] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:46:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:46:23.432388 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=68 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:46:23.531979 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=6 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:47:13.850529 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e170cbcc-d6bb-4db1-88f9-bfc2241c7620 service nova] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:47:13.863113 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e170cbcc-d6bb-4db1-88f9-bfc2241c7620 service nova] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:47:13.867365 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15e8e02-bb05-4d77-b2db-9ea9e8f6cc26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:47:13.868600 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15e8e02-bb05-4d77-b2db-9ea9e8f6cc26 service cinder] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:47:13.868947 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15e8e02-bb05-4d77-b2db-9ea9e8f6cc26 service cinder] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:47:13.869259 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15e8e02-bb05-4d77-b2db-9ea9e8f6cc26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:47:13.871137 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-c15e8e02-bb05-4d77-b2db-9ea9e8f6cc26 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:47:13.951524 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15e8e02-bb05-4d77-b2db-9ea9e8f6cc26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:47:13.953245 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15e8e02-bb05-4d77-b2db-9ea9e8f6cc26 service cinder] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:47:13.974157 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1204/2404] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:47:13 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:47:14.098803 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a972bc-9539-4f7f-ac26-12db42fd1bb4 service neutron] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:47:14.108813 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a972bc-9539-4f7f-ac26-12db42fd1bb4 service neutron] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:47:14.112860 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1155367a-c551-4550-9bef-be792e0569be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:47:14.113458 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1155367a-c551-4550-9bef-be792e0569be service cinder] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:47:14.113729 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1155367a-c551-4550-9bef-be792e0569be service cinder] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:47:14.114014 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1155367a-c551-4550-9bef-be792e0569be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:47:14.115949 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-1155367a-c551-4550-9bef-be792e0569be service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:47:14.198422 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1155367a-c551-4550-9bef-be792e0569be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:47:14.199897 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1155367a-c551-4550-9bef-be792e0569be service cinder] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:47:14.220150 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1201/2405] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:47:14 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:47:16.467731 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15e8e02-bb05-4d77-b2db-9ea9e8f6cc26 service cinder] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:47:16.544960 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15e8e02-bb05-4d77-b2db-9ea9e8f6cc26 service cinder] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:47:16.550313 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a15996-50e7-44a3-a2fa-7fab10f46e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:47:16.551162 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a15996-50e7-44a3-a2fa-7fab10f46e2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:47:16.551609 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a15996-50e7-44a3-a2fa-7fab10f46e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:47:16.552057 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a15996-50e7-44a3-a2fa-7fab10f46e2f admin admin] PATH_INFO: `/v3/users/765a46332add46a08dbcf383e1344549` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:47:16.553369 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a15996-50e7-44a3-a2fa-7fab10f46e2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=765a46332add46a08dbcf383e1344549)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:47:16.555327 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a15996-50e7-44a3-a2fa-7fab10f46e2f admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:47:16.607843 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-f6a15996-50e7-44a3-a2fa-7fab10f46e2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '765a46332add46a08dbcf383e1344549'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:47:16.608527 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-f6a15996-50e7-44a3-a2fa-7fab10f46e2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '765a46332add46a08dbcf383e1344549'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:47:16.616014 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-f6a15996-50e7-44a3-a2fa-7fab10f46e2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '765a46332add46a08dbcf383e1344549'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:47:16.628243 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-f6a15996-50e7-44a3-a2fa-7fab10f46e2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '765a46332add46a08dbcf383e1344549'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:47:16.628881 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-f6a15996-50e7-44a3-a2fa-7fab10f46e2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '765a46332add46a08dbcf383e1344549'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:47:16.638502 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:47:16.643183 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1205/2406] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:47:16 2022] DELETE /identity/v3/users/765a46332add46a08dbcf383e1344549 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:47:16.647968 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f447bbd2-44fe-4114-b3b6-982b6ff85e0b None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:47:16.648533 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f447bbd2-44fe-4114-b3b6-982b6ff85e0b None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:47:16.648638 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f447bbd2-44fe-4114-b3b6-982b6ff85e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:47:16.668590 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-f447bbd2-44fe-4114-b3b6-982b6ff85e0b None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:47:16.669785 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-f447bbd2-44fe-4114-b3b6-982b6ff85e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:47:16.729760 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1202/2407] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:47:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:47:16.739708 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a15996-50e7-44a3-a2fa-7fab10f46e2f admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:47:16.741167 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-f6a15996-50e7-44a3-a2fa-7fab10f46e2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:47:16.800134 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a15996-50e7-44a3-a2fa-7fab10f46e2f admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:47:16.803827 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebb60fc-a90a-43a1-85b8-2f7cdc954b08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:47:16.804429 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb60fc-a90a-43a1-85b8-2f7cdc954b08 service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:47:16.804712 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb60fc-a90a-43a1-85b8-2f7cdc954b08 service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:47:16.804989 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb60fc-a90a-43a1-85b8-2f7cdc954b08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:47:16.814222 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebb60fc-a90a-43a1-85b8-2f7cdc954b08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:47:16.815661 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebb60fc-a90a-43a1-85b8-2f7cdc954b08 service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:47:16.825039 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1206/2408] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:47:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:47:17.095365 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f447bbd2-44fe-4114-b3b6-982b6ff85e0b None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:47:17.096847 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-f447bbd2-44fe-4114-b3b6-982b6ff85e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:47:17.109693 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f447bbd2-44fe-4114-b3b6-982b6ff85e0b None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:47:17.114576 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0684ddac-0e24-416b-8744-c2638651ae2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:47:17.115210 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0684ddac-0e24-416b-8744-c2638651ae2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:47:17.115774 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0684ddac-0e24-416b-8744-c2638651ae2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:47:17.116107 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0684ddac-0e24-416b-8744-c2638651ae2c admin admin] PATH_INFO: `/v3/projects/039e29833028467786789a052a0b963e` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:47:17.117391 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0684ddac-0e24-416b-8744-c2638651ae2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=039e29833028467786789a052a0b963e)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:47:17.119528 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0684ddac-0e24-416b-8744-c2638651ae2c admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:47:17.169780 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-0684ddac-0e24-416b-8744-c2638651ae2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '039e29833028467786789a052a0b963e', 'request_id': 'req-0684ddac-0e24-416b-8744-c2638651ae2c'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:47:17.176733 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-0684ddac-0e24-416b-8744-c2638651ae2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '039e29833028467786789a052a0b963e', 'request_id': 'req-0684ddac-0e24-416b-8744-c2638651ae2c'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:47:17.182809 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-0684ddac-0e24-416b-8744-c2638651ae2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '039e29833028467786789a052a0b963e', 'request_id': 'req-0684ddac-0e24-416b-8744-c2638651ae2c'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:47:17.185448 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:47:17.199077 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-0684ddac-0e24-416b-8744-c2638651ae2c admin admin] The token cache is being invalidate because project 039e29833028467786789a052a0b963e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81618) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:47:17.199364 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-0684ddac-0e24-416b-8744-c2638651ae2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:47:17.200009 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-0684ddac-0e24-416b-8744-c2638651ae2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:47:17.201494 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1203/2409] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:47:17 2022] DELETE /identity/v3/projects/039e29833028467786789a052a0b963e => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:47:17.206597 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebb60fc-a90a-43a1-85b8-2f7cdc954b08 service neutron] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:47:17.216670 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebb60fc-a90a-43a1-85b8-2f7cdc954b08 service neutron] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:47:17.276134 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedbf4af-3664-4e47-b3b0-36ca0d74e9c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:47:17.276840 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedbf4af-3664-4e47-b3b0-36ca0d74e9c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:47:17.276965 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedbf4af-3664-4e47-b3b0-36ca0d74e9c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:47:17.277189 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedbf4af-3664-4e47-b3b0-36ca0d74e9c4 admin admin] PATH_INFO: `/v3/users/a60ebfbddb884708a303f1b9a439cbb9` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:47:17.278319 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedbf4af-3664-4e47-b3b0-36ca0d74e9c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a60ebfbddb884708a303f1b9a439cbb9)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:47:17.280279 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedbf4af-3664-4e47-b3b0-36ca0d74e9c4 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:47:17.339267 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-cedbf4af-3664-4e47-b3b0-36ca0d74e9c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a60ebfbddb884708a303f1b9a439cbb9'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:47:17.339938 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-cedbf4af-3664-4e47-b3b0-36ca0d74e9c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a60ebfbddb884708a303f1b9a439cbb9'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:47:17.348761 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-cedbf4af-3664-4e47-b3b0-36ca0d74e9c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a60ebfbddb884708a303f1b9a439cbb9'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:47:17.360949 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-cedbf4af-3664-4e47-b3b0-36ca0d74e9c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a60ebfbddb884708a303f1b9a439cbb9'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:47:17.361738 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-cedbf4af-3664-4e47-b3b0-36ca0d74e9c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a60ebfbddb884708a303f1b9a439cbb9'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:47:17.376143 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1207/2410] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:47:17 2022] DELETE /identity/v3/users/a60ebfbddb884708a303f1b9a439cbb9 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:47:17.659489 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0684ddac-0e24-416b-8744-c2638651ae2c admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:47:17.661047 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-0684ddac-0e24-416b-8744-c2638651ae2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:47:17.678636 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0684ddac-0e24-416b-8744-c2638651ae2c admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:47:17.745441 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143a443c-75f9-49f2-add5-80f2b2fa21db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:47:17.746094 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143a443c-75f9-49f2-add5-80f2b2fa21db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:47:17.746391 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143a443c-75f9-49f2-add5-80f2b2fa21db admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:47:17.746661 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143a443c-75f9-49f2-add5-80f2b2fa21db admin admin] PATH_INFO: `/v3/projects/9d56de62d4c14fbaaa1c6ab97eb41e4f` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:47:17.747901 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143a443c-75f9-49f2-add5-80f2b2fa21db admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d56de62d4c14fbaaa1c6ab97eb41e4f)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:47:17.749793 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143a443c-75f9-49f2-add5-80f2b2fa21db admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:47:17.797734 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-143a443c-75f9-49f2-add5-80f2b2fa21db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d56de62d4c14fbaaa1c6ab97eb41e4f', 'request_id': 'req-143a443c-75f9-49f2-add5-80f2b2fa21db'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:47:17.803880 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-143a443c-75f9-49f2-add5-80f2b2fa21db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d56de62d4c14fbaaa1c6ab97eb41e4f', 'request_id': 'req-143a443c-75f9-49f2-add5-80f2b2fa21db'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:47:17.809867 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-143a443c-75f9-49f2-add5-80f2b2fa21db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d56de62d4c14fbaaa1c6ab97eb41e4f', 'request_id': 'req-143a443c-75f9-49f2-add5-80f2b2fa21db'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:47:17.822781 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-143a443c-75f9-49f2-add5-80f2b2fa21db admin admin] The token cache is being invalidate because project 9d56de62d4c14fbaaa1c6ab97eb41e4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81618) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:47:17.823070 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-143a443c-75f9-49f2-add5-80f2b2fa21db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:47:17.823681 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-143a443c-75f9-49f2-add5-80f2b2fa21db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:47:17.825157 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1204/2411] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:47:17 2022] DELETE /identity/v3/projects/9d56de62d4c14fbaaa1c6ab97eb41e4f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:47:17.830221 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f4f948-17bd-4e29-9c55-b316ab347f2c None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:47:17.830772 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f4f948-17bd-4e29-9c55-b316ab347f2c None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:47:17.830772 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f4f948-17bd-4e29-9c55-b316ab347f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:47:17.850321 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-86f4f948-17bd-4e29-9c55-b316ab347f2c None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:47:17.851558 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-86f4f948-17bd-4e29-9c55-b316ab347f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:47:17.868698 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:47:17.911894 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1208/2412] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:47:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:47:17.919975 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654e5fc3-4d96-4911-860e-670c666c9504 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:47:17.920283 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654e5fc3-4d96-4911-860e-670c666c9504 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:47:17.920554 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654e5fc3-4d96-4911-860e-670c666c9504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:47:17.941199 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-654e5fc3-4d96-4911-860e-670c666c9504 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:47:17.942405 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-654e5fc3-4d96-4911-860e-670c666c9504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:47:17.950119 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1205/2413] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:47:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:47:17.954401 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f4f948-17bd-4e29-9c55-b316ab347f2c None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:47:17.964108 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f4f948-17bd-4e29-9c55-b316ab347f2c None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:47:17.968704 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd72015-bff1-41ba-bc77-b39f19c53c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:47:17.969296 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd72015-bff1-41ba-bc77-b39f19c53c67 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:47:17.969580 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd72015-bff1-41ba-bc77-b39f19c53c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:47:17.969884 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd72015-bff1-41ba-bc77-b39f19c53c67 admin admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:47:17.970554 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd72015-bff1-41ba-bc77-b39f19c53c67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:47:17.970861 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd72015-bff1-41ba-bc77-b39f19c53c67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:47:17.972443 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd72015-bff1-41ba-bc77-b39f19c53c67 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:47:17.983881 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1209/2414] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:47:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:47:17.988842 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654e5fc3-4d96-4911-860e-670c666c9504 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:47:17.997939 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654e5fc3-4d96-4911-860e-670c666c9504 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:47:18.003350 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590e511e-2d0f-426f-905d-c01520b2803a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:47:18.003948 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e511e-2d0f-426f-905d-c01520b2803a admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:47:18.004203 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e511e-2d0f-426f-905d-c01520b2803a admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:47:18.004495 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e511e-2d0f-426f-905d-c01520b2803a admin admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:47:18.005124 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590e511e-2d0f-426f-905d-c01520b2803a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:47:18.007033 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590e511e-2d0f-426f-905d-c01520b2803a admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:47:18.034870 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1206/2415] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:47:17 2022] POST /identity/v3/projects => generated 387 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:47:18.039413 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd72015-bff1-41ba-bc77-b39f19c53c67 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:47:18.049861 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd72015-bff1-41ba-bc77-b39f19c53c67 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:47:18.107747 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f6cef5-7af2-4d98-8b8f-5bf947654bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:47:18.108346 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f6cef5-7af2-4d98-8b8f-5bf947654bb4 admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:47:18.108620 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f6cef5-7af2-4d98-8b8f-5bf947654bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:47:18.108887 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f6cef5-7af2-4d98-8b8f-5bf947654bb4 admin admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:47:18.110033 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f6cef5-7af2-4d98-8b8f-5bf947654bb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:47:18.112001 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f6cef5-7af2-4d98-8b8f-5bf947654bb4 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:47:18.133241 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1210/2416] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:47:18 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 16:47:18.137444 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590e511e-2d0f-426f-905d-c01520b2803a admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:47:18.147593 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590e511e-2d0f-426f-905d-c01520b2803a admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:47:18.152243 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7208671f-6337-4ce2-a708-b53792785d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:47:18.152838 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208671f-6337-4ce2-a708-b53792785d1b admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:47:18.153120 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208671f-6337-4ce2-a708-b53792785d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:47:18.153395 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208671f-6337-4ce2-a708-b53792785d1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:47:18.154030 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7208671f-6337-4ce2-a708-b53792785d1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:47:18.155666 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7208671f-6337-4ce2-a708-b53792785d1b admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:47:18.166621 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1207/2417] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:47:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:47:18.170565 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f6cef5-7af2-4d98-8b8f-5bf947654bb4 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:47:18.182995 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f6cef5-7af2-4d98-8b8f-5bf947654bb4 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:47:18.187543 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1c2725-adc0-4f3a-8570-983364872767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:47:18.188138 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1c2725-adc0-4f3a-8570-983364872767 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:47:18.188417 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1c2725-adc0-4f3a-8570-983364872767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:47:18.188691 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1c2725-adc0-4f3a-8570-983364872767 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:47:18.189328 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1c2725-adc0-4f3a-8570-983364872767 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:47:18.190949 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1c2725-adc0-4f3a-8570-983364872767 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:47:18.201368 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1211/2418] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:47:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:47:18.205577 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7208671f-6337-4ce2-a708-b53792785d1b admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:47:18.215581 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7208671f-6337-4ce2-a708-b53792785d1b admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:47:18.221420 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003c260c-25a8-4516-a2ef-fa0940cd45ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:47:18.222094 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003c260c-25a8-4516-a2ef-fa0940cd45ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:47:18.222386 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003c260c-25a8-4516-a2ef-fa0940cd45ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:47:18.222671 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003c260c-25a8-4516-a2ef-fa0940cd45ab admin admin] PATH_INFO: `/v3/projects/37f2735f251b4cceab6880ea213f052d/users/f8df0d82b7cc40648249ba133f167741/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:47:18.237835 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003c260c-25a8-4516-a2ef-fa0940cd45ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=37f2735f251b4cceab6880ea213f052d, user_id=f8df0d82b7cc40648249ba133f167741, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:47:18.240484 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003c260c-25a8-4516-a2ef-fa0940cd45ab admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:47:18.253229 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1208/2419] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:47:18 2022] PUT /identity/v3/projects/37f2735f251b4cceab6880ea213f052d/users/f8df0d82b7cc40648249ba133f167741/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:47:18.323702 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e536b5-5d88-474a-8f90-e62aeaf2c8b4 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:47:18.323917 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e536b5-5d88-474a-8f90-e62aeaf2c8b4 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:47:18.324104 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e536b5-5d88-474a-8f90-e62aeaf2c8b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:47:18.351845 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-23e536b5-5d88-474a-8f90-e62aeaf2c8b4 None None] MFA Rules not processed for user `f8df0d82b7cc40648249ba133f167741`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:47:18.353060 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-23e536b5-5d88-474a-8f90-e62aeaf2c8b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:47:18.417277 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1212/2420] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:47:18 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:47:18.422454 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003c260c-25a8-4516-a2ef-fa0940cd45ab admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:47:18.443691 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:47:18.487683 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003c260c-25a8-4516-a2ef-fa0940cd45ab admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:47:18.492573 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0ddd42-5b97-4bcf-b68e-778eea9f3eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:47:18.493190 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ddd42-5b97-4bcf-b68e-778eea9f3eea admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:47:18.493476 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ddd42-5b97-4bcf-b68e-778eea9f3eea admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:47:18.493753 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ddd42-5b97-4bcf-b68e-778eea9f3eea admin admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:47:18.494411 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0ddd42-5b97-4bcf-b68e-778eea9f3eea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:47:18.496356 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0ddd42-5b97-4bcf-b68e-778eea9f3eea admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:47:18.526133 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1209/2421] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:47:18 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:47:18.530120 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e536b5-5d88-474a-8f90-e62aeaf2c8b4 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:47:18.544241 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e536b5-5d88-474a-8f90-e62aeaf2c8b4 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:47:18.560479 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:47:18.634300 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6b32fd-6860-4a85-995c-49c55cc9580b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:47:18.635082 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b32fd-6860-4a85-995c-49c55cc9580b admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:47:18.635468 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b32fd-6860-4a85-995c-49c55cc9580b admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:47:18.635846 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b32fd-6860-4a85-995c-49c55cc9580b admin admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:47:18.636610 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6b32fd-6860-4a85-995c-49c55cc9580b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:47:18.639018 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6b32fd-6860-4a85-995c-49c55cc9580b admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:47:18.670420 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1213/2422] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:47:18 2022] POST /identity/v3/users => generated 293 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 16:47:18.674051 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0ddd42-5b97-4bcf-b68e-778eea9f3eea admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:47:18.690065 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0ddd42-5b97-4bcf-b68e-778eea9f3eea admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:47:18.696426 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fca57d5-f2eb-4eff-9f8b-7d5bf63e122b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:47:18.697033 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca57d5-f2eb-4eff-9f8b-7d5bf63e122b admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:47:18.697366 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca57d5-f2eb-4eff-9f8b-7d5bf63e122b admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:47:18.697644 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca57d5-f2eb-4eff-9f8b-7d5bf63e122b admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:47:18.698339 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fca57d5-f2eb-4eff-9f8b-7d5bf63e122b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:47:18.700069 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fca57d5-f2eb-4eff-9f8b-7d5bf63e122b admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:47:18.714861 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1210/2423] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:47:18 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:47:18.718578 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6b32fd-6860-4a85-995c-49c55cc9580b admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:47:18.729289 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6b32fd-6860-4a85-995c-49c55cc9580b admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:47:18.734365 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b6aa17-997e-434f-b0a5-21f876981563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:47:18.735054 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b6aa17-997e-434f-b0a5-21f876981563 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:47:18.735407 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b6aa17-997e-434f-b0a5-21f876981563 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:47:18.735745 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b6aa17-997e-434f-b0a5-21f876981563 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:47:18.736456 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b6aa17-997e-434f-b0a5-21f876981563 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:47:18.738158 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b6aa17-997e-434f-b0a5-21f876981563 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:47:18.750451 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1214/2424] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:47:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:47:18.752789 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fca57d5-f2eb-4eff-9f8b-7d5bf63e122b admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:47:18.763318 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fca57d5-f2eb-4eff-9f8b-7d5bf63e122b admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:47:18.768576 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b01f74d-ebd1-4a08-aeba-2d95e9e9ab41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:47:18.769236 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b01f74d-ebd1-4a08-aeba-2d95e9e9ab41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:47:18.769518 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b01f74d-ebd1-4a08-aeba-2d95e9e9ab41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:47:18.769800 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b01f74d-ebd1-4a08-aeba-2d95e9e9ab41 admin admin] PATH_INFO: `/v3/projects/e09d48003d6247d48d36924a7ad4ec25/users/55af60a412d84b3f9a4fb511f8747402/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:47:18.785294 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b01f74d-ebd1-4a08-aeba-2d95e9e9ab41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e09d48003d6247d48d36924a7ad4ec25, user_id=55af60a412d84b3f9a4fb511f8747402, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:47:18.787963 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b01f74d-ebd1-4a08-aeba-2d95e9e9ab41 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:47:18.800828 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1211/2425] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:47:18 2022] PUT /identity/v3/projects/e09d48003d6247d48d36924a7ad4ec25/users/55af60a412d84b3f9a4fb511f8747402/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:47:18.811834 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b11382-ba17-4988-ba63-16c8cc1c3bb5 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:47:18.812119 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b11382-ba17-4988-ba63-16c8cc1c3bb5 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:47:18.812416 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b11382-ba17-4988-ba63-16c8cc1c3bb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:47:18.840585 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-69b11382-ba17-4988-ba63-16c8cc1c3bb5 None None] MFA Rules not processed for user `55af60a412d84b3f9a4fb511f8747402`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:47:18.841717 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-69b11382-ba17-4988-ba63-16c8cc1c3bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:47:18.905208 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1215/2426] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:47:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:47:18.909800 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b01f74d-ebd1-4a08-aeba-2d95e9e9ab41 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:47:18.973652 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b01f74d-ebd1-4a08-aeba-2d95e9e9ab41 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:47:18.978292 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee294c63-7206-4474-8d08-73e9175ac3c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:47:18.978922 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee294c63-7206-4474-8d08-73e9175ac3c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:47:18.979201 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee294c63-7206-4474-8d08-73e9175ac3c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:47:18.979479 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee294c63-7206-4474-8d08-73e9175ac3c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:47:18.980128 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee294c63-7206-4474-8d08-73e9175ac3c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:47:18.982014 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee294c63-7206-4474-8d08-73e9175ac3c6 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:47:19.009204 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1212/2427] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:47:18 2022] POST /identity/v3/projects => generated 387 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:47:19.015292 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b11382-ba17-4988-ba63-16c8cc1c3bb5 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:47:19.027241 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b11382-ba17-4988-ba63-16c8cc1c3bb5 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:47:19.089733 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b1fa55-c381-4345-8dba-a2c5034b6b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:47:19.090340 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b1fa55-c381-4345-8dba-a2c5034b6b6a admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:47:19.090716 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b1fa55-c381-4345-8dba-a2c5034b6b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:47:19.090949 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b1fa55-c381-4345-8dba-a2c5034b6b6a admin admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:47:19.091568 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b1fa55-c381-4345-8dba-a2c5034b6b6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:47:19.093487 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b1fa55-c381-4345-8dba-a2c5034b6b6a admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:47:19.115610 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1216/2428] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:47:19 2022] POST /identity/v3/users => generated 300 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:47:19.119777 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee294c63-7206-4474-8d08-73e9175ac3c6 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:47:19.130538 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee294c63-7206-4474-8d08-73e9175ac3c6 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:47:19.135114 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc489305-0907-4669-a83f-1a95193b27aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:47:19.135708 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc489305-0907-4669-a83f-1a95193b27aa admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:47:19.135989 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc489305-0907-4669-a83f-1a95193b27aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:47:19.136272 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc489305-0907-4669-a83f-1a95193b27aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:47:19.136908 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc489305-0907-4669-a83f-1a95193b27aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:47:19.138515 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc489305-0907-4669-a83f-1a95193b27aa admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:47:19.150559 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1213/2429] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:47:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:47:19.154602 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b1fa55-c381-4345-8dba-a2c5034b6b6a admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:47:19.164551 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b1fa55-c381-4345-8dba-a2c5034b6b6a admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:47:19.169337 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2146cb6-4a35-4d96-8868-f5046e0629b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:47:19.170019 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2146cb6-4a35-4d96-8868-f5046e0629b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:47:19.170297 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2146cb6-4a35-4d96-8868-f5046e0629b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:47:19.170605 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2146cb6-4a35-4d96-8868-f5046e0629b2 admin admin] PATH_INFO: `/v3/projects/614b1aa8141140aea0d6af28d123d9d9/users/40b74ebd3f8b40feac6edc7697dd98b3/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:47:19.182879 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:47:19.186136 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2146cb6-4a35-4d96-8868-f5046e0629b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=614b1aa8141140aea0d6af28d123d9d9, user_id=40b74ebd3f8b40feac6edc7697dd98b3, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:47:19.189935 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2146cb6-4a35-4d96-8868-f5046e0629b2 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:47:19.202964 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1217/2430] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:47:19 2022] PUT /identity/v3/projects/614b1aa8141140aea0d6af28d123d9d9/users/40b74ebd3f8b40feac6edc7697dd98b3/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:47:19.207539 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc489305-0907-4669-a83f-1a95193b27aa admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:47:19.217364 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc489305-0907-4669-a83f-1a95193b27aa admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:47:19.256938 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:47:19.273533 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fcd5b5-aeaa-4c3d-a713-5302a8664ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:47:19.274132 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fcd5b5-aeaa-4c3d-a713-5302a8664ca3 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:47:19.274434 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fcd5b5-aeaa-4c3d-a713-5302a8664ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:47:19.274702 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fcd5b5-aeaa-4c3d-a713-5302a8664ca3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:47:19.275325 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fcd5b5-aeaa-4c3d-a713-5302a8664ca3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:47:19.276891 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fcd5b5-aeaa-4c3d-a713-5302a8664ca3 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:47:19.286714 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1214/2431] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:47:19 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 16:47:19.293104 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2146cb6-4a35-4d96-8868-f5046e0629b2 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:47:19.304718 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2146cb6-4a35-4d96-8868-f5046e0629b2 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:47:19.309779 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6e86ec-f597-477a-8dd1-07c94028e9de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:47:19.310460 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6e86ec-f597-477a-8dd1-07c94028e9de admin admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:47:19.310744 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6e86ec-f597-477a-8dd1-07c94028e9de admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:47:19.311019 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6e86ec-f597-477a-8dd1-07c94028e9de admin admin] PATH_INFO: `/v3/projects/614b1aa8141140aea0d6af28d123d9d9/users/40b74ebd3f8b40feac6edc7697dd98b3/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:47:19.313523 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6e86ec-f597-477a-8dd1-07c94028e9de admin admin] RBAC: Authorizing `identity:create_grant(project_id=614b1aa8141140aea0d6af28d123d9d9, user_id=40b74ebd3f8b40feac6edc7697dd98b3, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:47:19.316178 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6e86ec-f597-477a-8dd1-07c94028e9de admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:47:19.326287 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1218/2432] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:47:19 2022] PUT /identity/v3/projects/614b1aa8141140aea0d6af28d123d9d9/users/40b74ebd3f8b40feac6edc7697dd98b3/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:47:19.337931 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864aa52-3bab-435d-8f17-9944ae486bea None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:47:19.338220 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864aa52-3bab-435d-8f17-9944ae486bea None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:47:19.338528 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864aa52-3bab-435d-8f17-9944ae486bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:47:19.366237 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-7864aa52-3bab-435d-8f17-9944ae486bea None None] MFA Rules not processed for user `40b74ebd3f8b40feac6edc7697dd98b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:47:19.367427 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-7864aa52-3bab-435d-8f17-9944ae486bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:47:19.440851 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1215/2433] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:47:19 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:47:19.453140 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6e86ec-f597-477a-8dd1-07c94028e9de admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:47:19.454946 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-3e6e86ec-f597-477a-8dd1-07c94028e9de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:47:19.512798 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6e86ec-f597-477a-8dd1-07c94028e9de admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:47:19.516552 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee5e68-4f23-443e-bb75-2135e254ee1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:47:19.517162 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee5e68-4f23-443e-bb75-2135e254ee1c service cinder] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:47:19.517441 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee5e68-4f23-443e-bb75-2135e254ee1c service cinder] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:47:19.517780 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee5e68-4f23-443e-bb75-2135e254ee1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:47:19.527676 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eee5e68-4f23-443e-bb75-2135e254ee1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:47:19.529157 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eee5e68-4f23-443e-bb75-2135e254ee1c service cinder] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:47:19.538762 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1219/2434] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:47:19 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:47:20.210019 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7864aa52-3bab-435d-8f17-9944ae486bea None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:47:20.219582 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7864aa52-3bab-435d-8f17-9944ae486bea None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:47:20.223121 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf6abe9-85e4-41b0-b596-f80a51f587e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:47:20.223708 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf6abe9-85e4-41b0-b596-f80a51f587e4 service cinder] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:47:20.223981 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf6abe9-85e4-41b0-b596-f80a51f587e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:47:20.224248 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf6abe9-85e4-41b0-b596-f80a51f587e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:47:20.279342 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf6abe9-85e4-41b0-b596-f80a51f587e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:47:20.280741 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf6abe9-85e4-41b0-b596-f80a51f587e4 service cinder] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:47:20.289508 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1216/2435] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:47:20 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:47:29.534654 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=34,keystone:INSERT=2 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:47:30.285504 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=71 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:48:03.729359 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee5e68-4f23-443e-bb75-2135e254ee1c service cinder] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:48:03.731359 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-4eee5e68-4f23-443e-bb75-2135e254ee1c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:48:03.808603 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee5e68-4f23-443e-bb75-2135e254ee1c service cinder] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:48:03.819945 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0371383-a880-4af6-b175-883b984814c7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1a985495ed48a4acc48e2e6fc7f977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:48:03.820428 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0371383-a880-4af6-b175-883b984814c7 service placement] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:48:03.820706 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0371383-a880-4af6-b175-883b984814c7 service placement] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:48:03.820976 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0371383-a880-4af6-b175-883b984814c7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:48:03.822755 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-b0371383-a880-4af6-b175-883b984814c7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:48:03.835185 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0371383-a880-4af6-b175-883b984814c7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:48:03.836739 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0371383-a880-4af6-b175-883b984814c7 service placement] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:48:03.853288 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1220/2436] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:48:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:48:13.140573 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf6abe9-85e4-41b0-b596-f80a51f587e4 service cinder] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:48:13.142050 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-9cf6abe9-85e4-41b0-b596-f80a51f587e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:48:13.198483 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf6abe9-85e4-41b0-b596-f80a51f587e4 service cinder] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:48:13.202066 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff00a45-5256-4fb8-9415-364481ba6b35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:48:13.202697 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff00a45-5256-4fb8-9415-364481ba6b35 service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:48:13.202974 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff00a45-5256-4fb8-9415-364481ba6b35 service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:48:13.203247 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff00a45-5256-4fb8-9415-364481ba6b35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:48:13.205483 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-9ff00a45-5256-4fb8-9415-364481ba6b35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:48:13.275286 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff00a45-5256-4fb8-9415-364481ba6b35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:48:13.276701 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff00a45-5256-4fb8-9415-364481ba6b35 service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:48:13.288262 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1217/2437] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:48:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:48:13.847085 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=32 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:48:18.795287 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0371383-a880-4af6-b175-883b984814c7 service placement] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:48:18.805034 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0371383-a880-4af6-b175-883b984814c7 service placement] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:48:18.808998 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23431e0-2ce2-4981-9a4f-3227dd09dd50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:48:18.809587 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23431e0-2ce2-4981-9a4f-3227dd09dd50 service cinder] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:48:18.809870 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23431e0-2ce2-4981-9a4f-3227dd09dd50 service cinder] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:48:18.810145 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23431e0-2ce2-4981-9a4f-3227dd09dd50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:48:18.811911 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-a23431e0-2ce2-4981-9a4f-3227dd09dd50 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:48:18.890598 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23431e0-2ce2-4981-9a4f-3227dd09dd50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:48:18.891939 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23431e0-2ce2-4981-9a4f-3227dd09dd50 service cinder] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:48:18.910427 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1221/2438] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:48:18 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:48:23.282736 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=54 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:48:28.907245 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=41 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:04.374121 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff00a45-5256-4fb8-9415-364481ba6b35 service neutron] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:04.384052 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff00a45-5256-4fb8-9415-364481ba6b35 service neutron] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:04.388074 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a458b5b3-cac2-4e00-af39-cc2880eead24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:04.388675 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a458b5b3-cac2-4e00-af39-cc2880eead24 service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:04.388950 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a458b5b3-cac2-4e00-af39-cc2880eead24 service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:04.389221 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a458b5b3-cac2-4e00-af39-cc2880eead24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:04.390970 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-a458b5b3-cac2-4e00-af39-cc2880eead24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:04.460921 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a458b5b3-cac2-4e00-af39-cc2880eead24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:04.462482 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a458b5b3-cac2-4e00-af39-cc2880eead24 service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:04.472295 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1218/2439] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:49:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:49:05.988267 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23431e0-2ce2-4981-9a4f-3227dd09dd50 service cinder] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:05.990914 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-a23431e0-2ce2-4981-9a4f-3227dd09dd50 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:06.026009 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23431e0-2ce2-4981-9a4f-3227dd09dd50 service cinder] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:06.034479 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c56a74-fb4a-4963-9fbf-36d9d7910b72 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:06.035589 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c56a74-fb4a-4963-9fbf-36d9d7910b72 service nova] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:06.036106 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c56a74-fb4a-4963-9fbf-36d9d7910b72 service nova] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:06.036603 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c56a74-fb4a-4963-9fbf-36d9d7910b72 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:06.039600 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-25c56a74-fb4a-4963-9fbf-36d9d7910b72 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:06.084350 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c56a74-fb4a-4963-9fbf-36d9d7910b72 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:06.086132 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c56a74-fb4a-4963-9fbf-36d9d7910b72 service nova] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:06.105069 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1222/2440] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:49:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:49:07.203773 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a458b5b3-cac2-4e00-af39-cc2880eead24 service neutron] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:07.206894 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-a458b5b3-cac2-4e00-af39-cc2880eead24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:07.220773 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a458b5b3-cac2-4e00-af39-cc2880eead24 service neutron] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:07.224782 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc88522-dc49-4b1f-bcbd-28b807c7204e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:07.227036 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc88522-dc49-4b1f-bcbd-28b807c7204e service cinder] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:07.227693 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc88522-dc49-4b1f-bcbd-28b807c7204e service cinder] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:07.228251 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc88522-dc49-4b1f-bcbd-28b807c7204e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:07.304511 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc88522-dc49-4b1f-bcbd-28b807c7204e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:07.307065 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc88522-dc49-4b1f-bcbd-28b807c7204e service cinder] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:07.331700 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1219/2441] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:49:07 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:49:09.534064 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c56a74-fb4a-4963-9fbf-36d9d7910b72 service nova] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:09.535550 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-25c56a74-fb4a-4963-9fbf-36d9d7910b72 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:09.552411 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c56a74-fb4a-4963-9fbf-36d9d7910b72 service nova] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:09.558362 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629f717b-8c51-43e6-b5cf-f3e34696419d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1a985495ed48a4acc48e2e6fc7f977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:09.559214 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629f717b-8c51-43e6-b5cf-f3e34696419d service placement] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:09.559653 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629f717b-8c51-43e6-b5cf-f3e34696419d service placement] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:09.560026 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629f717b-8c51-43e6-b5cf-f3e34696419d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:09.561883 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-629f717b-8c51-43e6-b5cf-f3e34696419d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:09.573816 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629f717b-8c51-43e6-b5cf-f3e34696419d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:09.575431 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629f717b-8c51-43e6-b5cf-f3e34696419d service placement] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:09.588795 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1223/2442] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:49:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:49:09.670334 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc88522-dc49-4b1f-bcbd-28b807c7204e service cinder] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:09.671782 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-fbc88522-dc49-4b1f-bcbd-28b807c7204e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:09.688717 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc88522-dc49-4b1f-bcbd-28b807c7204e service cinder] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:09.694241 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa0c787-4321-4417-b0a9-7bc463bf4db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:09.695754 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa0c787-4321-4417-b0a9-7bc463bf4db5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:09.696148 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa0c787-4321-4417-b0a9-7bc463bf4db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:09.696696 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa0c787-4321-4417-b0a9-7bc463bf4db5 admin admin] PATH_INFO: `/v3/users/754c1b5cb47746cf93c52d6dc0b8fa01` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:09.697948 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa0c787-4321-4417-b0a9-7bc463bf4db5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=754c1b5cb47746cf93c52d6dc0b8fa01)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:09.700740 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa0c787-4321-4417-b0a9-7bc463bf4db5 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:09.764414 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-5fa0c787-4321-4417-b0a9-7bc463bf4db5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '754c1b5cb47746cf93c52d6dc0b8fa01'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:09.773853 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-5fa0c787-4321-4417-b0a9-7bc463bf4db5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '754c1b5cb47746cf93c52d6dc0b8fa01'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:09.795215 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-5fa0c787-4321-4417-b0a9-7bc463bf4db5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '754c1b5cb47746cf93c52d6dc0b8fa01'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:09.806874 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-5fa0c787-4321-4417-b0a9-7bc463bf4db5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '754c1b5cb47746cf93c52d6dc0b8fa01'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:09.807540 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-5fa0c787-4321-4417-b0a9-7bc463bf4db5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '754c1b5cb47746cf93c52d6dc0b8fa01'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:09.822724 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1220/2443] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:49:09 2022] DELETE /identity/v3/users/754c1b5cb47746cf93c52d6dc0b8fa01 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:49:10.011560 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629f717b-8c51-43e6-b5cf-f3e34696419d service placement] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:10.023916 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629f717b-8c51-43e6-b5cf-f3e34696419d service placement] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:10.098255 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4df0c10-0a0a-4100-8891-040df1c1947a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:10.099170 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df0c10-0a0a-4100-8891-040df1c1947a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:10.099561 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df0c10-0a0a-4100-8891-040df1c1947a admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:10.099714 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df0c10-0a0a-4100-8891-040df1c1947a admin admin] PATH_INFO: `/v3/projects/23c705bc001e4228bdfcdfa916a368e9` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:10.100910 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4df0c10-0a0a-4100-8891-040df1c1947a admin admin] RBAC: Authorizing `identity:delete_project(project_id=23c705bc001e4228bdfcdfa916a368e9)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:10.102952 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4df0c10-0a0a-4100-8891-040df1c1947a admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:10.144900 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa0c787-4321-4417-b0a9-7bc463bf4db5 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:10.146420 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-5fa0c787-4321-4417-b0a9-7bc463bf4db5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:10.155684 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:10.160882 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa0c787-4321-4417-b0a9-7bc463bf4db5 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:10.164332 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-b4df0c10-0a0a-4100-8891-040df1c1947a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23c705bc001e4228bdfcdfa916a368e9', 'request_id': 'req-b4df0c10-0a0a-4100-8891-040df1c1947a'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:10.170964 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-b4df0c10-0a0a-4100-8891-040df1c1947a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23c705bc001e4228bdfcdfa916a368e9', 'request_id': 'req-b4df0c10-0a0a-4100-8891-040df1c1947a'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:10.176670 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-b4df0c10-0a0a-4100-8891-040df1c1947a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23c705bc001e4228bdfcdfa916a368e9', 'request_id': 'req-b4df0c10-0a0a-4100-8891-040df1c1947a'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:10.190883 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-b4df0c10-0a0a-4100-8891-040df1c1947a admin admin] The token cache is being invalidate because project 23c705bc001e4228bdfcdfa916a368e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81617) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:49:10.191167 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-b4df0c10-0a0a-4100-8891-040df1c1947a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:10.191811 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-b4df0c10-0a0a-4100-8891-040df1c1947a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:10.193475 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1224/2444] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:49:10 2022] DELETE /identity/v3/projects/23c705bc001e4228bdfcdfa916a368e9 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:49:10.199548 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd5549c-76bd-45a9-89ac-deccfb917bdd None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:10.199825 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd5549c-76bd-45a9-89ac-deccfb917bdd None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:10.200123 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd5549c-76bd-45a9-89ac-deccfb917bdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:10.219753 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-2bd5549c-76bd-45a9-89ac-deccfb917bdd None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:49:10.220951 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-2bd5549c-76bd-45a9-89ac-deccfb917bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:10.224581 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc6883b-b5e5-4359-a1cc-10e5414dcd1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:10.225218 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6883b-b5e5-4359-a1cc-10e5414dcd1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:10.225501 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6883b-b5e5-4359-a1cc-10e5414dcd1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:10.225855 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6883b-b5e5-4359-a1cc-10e5414dcd1f admin admin] PATH_INFO: `/v3/users/f8df0d82b7cc40648249ba133f167741` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:10.227075 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc6883b-b5e5-4359-a1cc-10e5414dcd1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8df0d82b7cc40648249ba133f167741)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:10.228956 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1225/2445] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:49:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:49:10.229142 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc6883b-b5e5-4359-a1cc-10e5414dcd1f admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:10.237130 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e43924-8e38-44c5-8fe5-87b7661b0111 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:10.237413 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e43924-8e38-44c5-8fe5-87b7661b0111 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:10.237682 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e43924-8e38-44c5-8fe5-87b7661b0111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:10.259485 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-71e43924-8e38-44c5-8fe5-87b7661b0111 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:49:10.260687 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-71e43924-8e38-44c5-8fe5-87b7661b0111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:10.270320 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1226/2446] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:49:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:49:10.275335 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e43924-8e38-44c5-8fe5-87b7661b0111 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:10.284419 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e43924-8e38-44c5-8fe5-87b7661b0111 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:10.287167 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-1dc6883b-b5e5-4359-a1cc-10e5414dcd1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8df0d82b7cc40648249ba133f167741'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:10.288956 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d014707b-8be7-4706-b31a-5105001b550e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:10.289544 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d014707b-8be7-4706-b31a-5105001b550e admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:10.289824 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d014707b-8be7-4706-b31a-5105001b550e admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:10.290091 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d014707b-8be7-4706-b31a-5105001b550e admin admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:10.290766 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d014707b-8be7-4706-b31a-5105001b550e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:49:10.291054 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d014707b-8be7-4706-b31a-5105001b550e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:10.292628 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d014707b-8be7-4706-b31a-5105001b550e admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:10.294629 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-1dc6883b-b5e5-4359-a1cc-10e5414dcd1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8df0d82b7cc40648249ba133f167741'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:10.305412 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1227/2447] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:49:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:49:10.308015 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-1dc6883b-b5e5-4359-a1cc-10e5414dcd1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8df0d82b7cc40648249ba133f167741'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:10.309972 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d014707b-8be7-4706-b31a-5105001b550e admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:10.315651 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-1dc6883b-b5e5-4359-a1cc-10e5414dcd1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8df0d82b7cc40648249ba133f167741'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:10.316278 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-1dc6883b-b5e5-4359-a1cc-10e5414dcd1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8df0d82b7cc40648249ba133f167741'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:10.321447 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d014707b-8be7-4706-b31a-5105001b550e admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:10.323097 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1221/2448] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:49:10 2022] DELETE /identity/v3/users/f8df0d82b7cc40648249ba133f167741 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 16:49:10.327514 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea7e23-9123-4b89-ac65-2a3517e4ff61 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:10.327869 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea7e23-9123-4b89-ac65-2a3517e4ff61 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:10.329673 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea7e23-9123-4b89-ac65-2a3517e4ff61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:10.350894 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-63ea7e23-9123-4b89-ac65-2a3517e4ff61 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:49:10.352198 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-63ea7e23-9123-4b89-ac65-2a3517e4ff61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:10.363677 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:10.400350 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0683b33b-d8a2-47fc-b864-826138e4e75e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:10.401366 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0683b33b-d8a2-47fc-b864-826138e4e75e admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:10.401755 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0683b33b-d8a2-47fc-b864-826138e4e75e admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:10.402030 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0683b33b-d8a2-47fc-b864-826138e4e75e admin admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:10.402704 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0683b33b-d8a2-47fc-b864-826138e4e75e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:10.404662 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0683b33b-d8a2-47fc-b864-826138e4e75e admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:10.421363 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1222/2449] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:49:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:49:10.431982 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ea7e23-9123-4b89-ac65-2a3517e4ff61 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:10.433593 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-63ea7e23-9123-4b89-ac65-2a3517e4ff61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:10.437014 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1228/2450] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:49:10 2022] POST /identity/v3/projects => generated 373 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:49:10.437975 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0683b33b-d8a2-47fc-b864-826138e4e75e admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:10.439605 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-0683b33b-d8a2-47fc-b864-826138e4e75e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:10.461296 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:10.496492 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ea7e23-9123-4b89-ac65-2a3517e4ff61 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:10.510897 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0683b33b-d8a2-47fc-b864-826138e4e75e admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:10.515680 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dad210-a441-48d1-be66-d1f673213f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:10.516294 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dad210-a441-48d1-be66-d1f673213f75 admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:10.516578 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dad210-a441-48d1-be66-d1f673213f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:10.516856 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dad210-a441-48d1-be66-d1f673213f75 admin admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:10.517494 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dad210-a441-48d1-be66-d1f673213f75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:10.519614 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dad210-a441-48d1-be66-d1f673213f75 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:10.543380 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1229/2451] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:49:10 2022] POST /identity/v3/users => generated 287 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 16:49:10.547418 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dad210-a441-48d1-be66-d1f673213f75 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:10.559025 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a95b9e6-9df8-4bea-8935-28562cf095ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:10.559025 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a95b9e6-9df8-4bea-8935-28562cf095ae service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:10.559415 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a95b9e6-9df8-4bea-8935-28562cf095ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:10.559415 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a95b9e6-9df8-4bea-8935-28562cf095ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:10.559927 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dad210-a441-48d1-be66-d1f673213f75 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:10.565068 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdfa607-5710-4869-9bf1-e5136aa32372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:10.565676 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdfa607-5710-4869-9bf1-e5136aa32372 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:10.565958 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdfa607-5710-4869-9bf1-e5136aa32372 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:10.567002 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdfa607-5710-4869-9bf1-e5136aa32372 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:10.567002 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdfa607-5710-4869-9bf1-e5136aa32372 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:10.568503 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdfa607-5710-4869-9bf1-e5136aa32372 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:10.572504 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a95b9e6-9df8-4bea-8935-28562cf095ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:10.574081 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a95b9e6-9df8-4bea-8935-28562cf095ae service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:10.580227 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1230/2452] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:49:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:49:10.584131 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdfa607-5710-4869-9bf1-e5136aa32372 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:10.589953 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1223/2453] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:49:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 16:49:10.595221 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdfa607-5710-4869-9bf1-e5136aa32372 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:10.600447 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f70f8a-cb27-454f-9804-66b142936ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:10.601568 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f70f8a-cb27-454f-9804-66b142936ce4 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:10.601843 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f70f8a-cb27-454f-9804-66b142936ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:10.602113 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f70f8a-cb27-454f-9804-66b142936ce4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:10.602785 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f70f8a-cb27-454f-9804-66b142936ce4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:10.604416 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f70f8a-cb27-454f-9804-66b142936ce4 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:10.615622 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1231/2454] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:49:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 16:49:10.620117 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a95b9e6-9df8-4bea-8935-28562cf095ae service neutron] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:10.632782 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a95b9e6-9df8-4bea-8935-28562cf095ae service neutron] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:10.638126 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0602bdce-be73-42b3-80ca-c1e2704c8217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:10.638912 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0602bdce-be73-42b3-80ca-c1e2704c8217 admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:10.641672 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0602bdce-be73-42b3-80ca-c1e2704c8217 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:10.641672 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0602bdce-be73-42b3-80ca-c1e2704c8217 admin admin] PATH_INFO: `/v3/projects/b27fc770f9ce456ebfbf373a9836f3e9/users/62767f55b93b4f469afa71a7d55ead34/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:10.662562 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0602bdce-be73-42b3-80ca-c1e2704c8217 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b27fc770f9ce456ebfbf373a9836f3e9, user_id=62767f55b93b4f469afa71a7d55ead34, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:10.665739 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0602bdce-be73-42b3-80ca-c1e2704c8217 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:10.690615 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1224/2455] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:49:10 2022] PUT /identity/v3/projects/b27fc770f9ce456ebfbf373a9836f3e9/users/62767f55b93b4f469afa71a7d55ead34/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:49:10.702740 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f360782f-5978-4852-b80f-3fa37509954f None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:10.703029 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f360782f-5978-4852-b80f-3fa37509954f None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:10.703307 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f360782f-5978-4852-b80f-3fa37509954f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:10.740019 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-f360782f-5978-4852-b80f-3fa37509954f None None] MFA Rules not processed for user `62767f55b93b4f469afa71a7d55ead34`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:49:10.742099 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-f360782f-5978-4852-b80f-3fa37509954f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:10.822004 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1232/2456] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:49:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:49:10.833014 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0602bdce-be73-42b3-80ca-c1e2704c8217 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:10.835423 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-0602bdce-be73-42b3-80ca-c1e2704c8217 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:10.917808 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0602bdce-be73-42b3-80ca-c1e2704c8217 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:10.923877 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b230d1b5-11fb-4df2-820c-d5345eae3e75 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:10.924630 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b230d1b5-11fb-4df2-820c-d5345eae3e75 service nova] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:10.924960 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b230d1b5-11fb-4df2-820c-d5345eae3e75 service nova] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:10.926168 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b230d1b5-11fb-4df2-820c-d5345eae3e75 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:10.937879 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b230d1b5-11fb-4df2-820c-d5345eae3e75 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:10.939527 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b230d1b5-11fb-4df2-820c-d5345eae3e75 service nova] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:10.950051 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1225/2457] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:49:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 16:49:10.954928 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f360782f-5978-4852-b80f-3fa37509954f None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:10.956543 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-f360782f-5978-4852-b80f-3fa37509954f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:10.990016 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:11.052863 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f360782f-5978-4852-b80f-3fa37509954f None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:11.058165 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07c82aa-72f1-4a34-b506-9f6281cd5305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:11.058829 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c82aa-72f1-4a34-b506-9f6281cd5305 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:11.059109 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c82aa-72f1-4a34-b506-9f6281cd5305 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:11.059382 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c82aa-72f1-4a34-b506-9f6281cd5305 admin admin] PATH_INFO: `/v3/projects/37f2735f251b4cceab6880ea213f052d` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:11.060507 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07c82aa-72f1-4a34-b506-9f6281cd5305 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37f2735f251b4cceab6880ea213f052d)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:11.062429 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07c82aa-72f1-4a34-b506-9f6281cd5305 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:11.129824 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-e07c82aa-72f1-4a34-b506-9f6281cd5305 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37f2735f251b4cceab6880ea213f052d', 'request_id': 'req-e07c82aa-72f1-4a34-b506-9f6281cd5305'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:11.136990 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-e07c82aa-72f1-4a34-b506-9f6281cd5305 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37f2735f251b4cceab6880ea213f052d', 'request_id': 'req-e07c82aa-72f1-4a34-b506-9f6281cd5305'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:11.151796 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-e07c82aa-72f1-4a34-b506-9f6281cd5305 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37f2735f251b4cceab6880ea213f052d', 'request_id': 'req-e07c82aa-72f1-4a34-b506-9f6281cd5305'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:11.158766 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-e07c82aa-72f1-4a34-b506-9f6281cd5305 admin admin] The token cache is being invalidate because project 37f2735f251b4cceab6880ea213f052d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81617) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:49:11.159055 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-e07c82aa-72f1-4a34-b506-9f6281cd5305 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:11.159856 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-e07c82aa-72f1-4a34-b506-9f6281cd5305 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:11.161320 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1233/2458] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:49:10 2022] DELETE /identity/v3/projects/37f2735f251b4cceab6880ea213f052d => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:49:11.165330 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b230d1b5-11fb-4df2-820c-d5345eae3e75 service nova] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:11.175615 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b230d1b5-11fb-4df2-820c-d5345eae3e75 service nova] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:11.196963 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07c82aa-72f1-4a34-b506-9f6281cd5305 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:11.198636 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-e07c82aa-72f1-4a34-b506-9f6281cd5305 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:11.217920 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:11.234281 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ff66d1-e113-4489-b87c-52d344ffc1c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:11.235196 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ff66d1-e113-4489-b87c-52d344ffc1c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:11.235573 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ff66d1-e113-4489-b87c-52d344ffc1c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:11.236797 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ff66d1-e113-4489-b87c-52d344ffc1c5 admin admin] PATH_INFO: `/v3/users/55af60a412d84b3f9a4fb511f8747402` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:11.237965 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ff66d1-e113-4489-b87c-52d344ffc1c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55af60a412d84b3f9a4fb511f8747402)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:11.239921 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ff66d1-e113-4489-b87c-52d344ffc1c5 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:11.266584 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07c82aa-72f1-4a34-b506-9f6281cd5305 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:11.271428 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf3985c-0f56-4492-8ff1-69496438284d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:11.272089 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf3985c-0f56-4492-8ff1-69496438284d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:11.272421 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf3985c-0f56-4492-8ff1-69496438284d admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:11.273380 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf3985c-0f56-4492-8ff1-69496438284d admin admin] PATH_INFO: `/v3/users/62767f55b93b4f469afa71a7d55ead34` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:11.274995 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf3985c-0f56-4492-8ff1-69496438284d admin admin] RBAC: Authorizing `identity:delete_user(user_id=62767f55b93b4f469afa71a7d55ead34)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:11.277117 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf3985c-0f56-4492-8ff1-69496438284d admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:11.289543 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-48ff66d1-e113-4489-b87c-52d344ffc1c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55af60a412d84b3f9a4fb511f8747402'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:11.296810 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-48ff66d1-e113-4489-b87c-52d344ffc1c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55af60a412d84b3f9a4fb511f8747402'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:11.310591 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-48ff66d1-e113-4489-b87c-52d344ffc1c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55af60a412d84b3f9a4fb511f8747402'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:11.319208 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-48ff66d1-e113-4489-b87c-52d344ffc1c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55af60a412d84b3f9a4fb511f8747402'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:11.321397 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-48ff66d1-e113-4489-b87c-52d344ffc1c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55af60a412d84b3f9a4fb511f8747402'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:11.331661 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1226/2459] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:49:11 2022] DELETE /identity/v3/users/55af60a412d84b3f9a4fb511f8747402 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 16:49:11.341723 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-faf3985c-0f56-4492-8ff1-69496438284d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62767f55b93b4f469afa71a7d55ead34'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:11.342484 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-faf3985c-0f56-4492-8ff1-69496438284d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62767f55b93b4f469afa71a7d55ead34'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:11.348576 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:11.352585 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-faf3985c-0f56-4492-8ff1-69496438284d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62767f55b93b4f469afa71a7d55ead34'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:11.369494 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-faf3985c-0f56-4492-8ff1-69496438284d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62767f55b93b4f469afa71a7d55ead34'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:11.370847 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-faf3985c-0f56-4492-8ff1-69496438284d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62767f55b93b4f469afa71a7d55ead34'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:11.385640 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1234/2460] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:49:11 2022] DELETE /identity/v3/users/62767f55b93b4f469afa71a7d55ead34 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 16:49:11.390285 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d7eac-ecbe-4ddf-ba3d-74a44675bafe None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:11.390622 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d7eac-ecbe-4ddf-ba3d-74a44675bafe None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:11.390946 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d7eac-ecbe-4ddf-ba3d-74a44675bafe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:11.413553 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-9f0d7eac-ecbe-4ddf-ba3d-74a44675bafe None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:49:11.414779 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-9f0d7eac-ecbe-4ddf-ba3d-74a44675bafe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:11.486793 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1227/2461] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:49:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:49:11.501003 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf3985c-0f56-4492-8ff1-69496438284d admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:11.502787 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-faf3985c-0f56-4492-8ff1-69496438284d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:11.565354 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf3985c-0f56-4492-8ff1-69496438284d admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:11.569229 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711d57fc-4286-4a86-bb55-0da7631d6cfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:11.570427 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d57fc-4286-4a86-bb55-0da7631d6cfb service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:11.570710 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d57fc-4286-4a86-bb55-0da7631d6cfb service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:11.570984 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d57fc-4286-4a86-bb55-0da7631d6cfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:11.582016 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711d57fc-4286-4a86-bb55-0da7631d6cfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:11.583421 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711d57fc-4286-4a86-bb55-0da7631d6cfb service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:11.594129 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1235/2462] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:49:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 16:49:11.649334 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0d7eac-ecbe-4ddf-ba3d-74a44675bafe None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:11.651490 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-9f0d7eac-ecbe-4ddf-ba3d-74a44675bafe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:11.675793 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0d7eac-ecbe-4ddf-ba3d-74a44675bafe None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:11.689529 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d909d4b9-bc17-415d-889a-e214ac8e8e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:11.690245 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d909d4b9-bc17-415d-889a-e214ac8e8e5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:11.690565 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d909d4b9-bc17-415d-889a-e214ac8e8e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:11.690840 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d909d4b9-bc17-415d-889a-e214ac8e8e5d admin admin] PATH_INFO: `/v3/projects/e09d48003d6247d48d36924a7ad4ec25` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:11.692010 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d909d4b9-bc17-415d-889a-e214ac8e8e5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e09d48003d6247d48d36924a7ad4ec25)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:11.693914 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d909d4b9-bc17-415d-889a-e214ac8e8e5d admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:11.749525 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-d909d4b9-bc17-415d-889a-e214ac8e8e5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e09d48003d6247d48d36924a7ad4ec25', 'request_id': 'req-d909d4b9-bc17-415d-889a-e214ac8e8e5d'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:11.756420 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-d909d4b9-bc17-415d-889a-e214ac8e8e5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e09d48003d6247d48d36924a7ad4ec25', 'request_id': 'req-d909d4b9-bc17-415d-889a-e214ac8e8e5d'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:11.766155 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-d909d4b9-bc17-415d-889a-e214ac8e8e5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e09d48003d6247d48d36924a7ad4ec25', 'request_id': 'req-d909d4b9-bc17-415d-889a-e214ac8e8e5d'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:11.770632 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:11.797154 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-d909d4b9-bc17-415d-889a-e214ac8e8e5d admin admin] The token cache is being invalidate because project e09d48003d6247d48d36924a7ad4ec25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81618) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:49:11.797538 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-d909d4b9-bc17-415d-889a-e214ac8e8e5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:11.798408 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-d909d4b9-bc17-415d-889a-e214ac8e8e5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:11.804969 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1228/2463] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:49:11 2022] DELETE /identity/v3/projects/e09d48003d6247d48d36924a7ad4ec25 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:49:11.806480 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711d57fc-4286-4a86-bb55-0da7631d6cfb service neutron] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:11.818592 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711d57fc-4286-4a86-bb55-0da7631d6cfb service neutron] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:11.895867 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1961c0da-7847-406f-944b-5a1d053fce1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:11.896657 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1961c0da-7847-406f-944b-5a1d053fce1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:11.897240 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1961c0da-7847-406f-944b-5a1d053fce1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:11.897763 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1961c0da-7847-406f-944b-5a1d053fce1d admin admin] PATH_INFO: `/v3/users/40b74ebd3f8b40feac6edc7697dd98b3` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:11.899537 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1961c0da-7847-406f-944b-5a1d053fce1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=40b74ebd3f8b40feac6edc7697dd98b3)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:11.901866 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1961c0da-7847-406f-944b-5a1d053fce1d admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:11.981879 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-1961c0da-7847-406f-944b-5a1d053fce1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40b74ebd3f8b40feac6edc7697dd98b3'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:11.982976 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-1961c0da-7847-406f-944b-5a1d053fce1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40b74ebd3f8b40feac6edc7697dd98b3'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:12.004563 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-1961c0da-7847-406f-944b-5a1d053fce1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40b74ebd3f8b40feac6edc7697dd98b3'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:12.010598 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-1961c0da-7847-406f-944b-5a1d053fce1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40b74ebd3f8b40feac6edc7697dd98b3'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:12.011579 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-1961c0da-7847-406f-944b-5a1d053fce1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40b74ebd3f8b40feac6edc7697dd98b3'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:12.013510 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:12.028821 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1236/2464] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:49:11 2022] DELETE /identity/v3/users/40b74ebd3f8b40feac6edc7697dd98b3 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:49:12.235151 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d909d4b9-bc17-415d-889a-e214ac8e8e5d admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:12.236738 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-d909d4b9-bc17-415d-889a-e214ac8e8e5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:12.251535 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d909d4b9-bc17-415d-889a-e214ac8e8e5d admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:12.337733 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1af020-e46d-4e98-a447-7749d7fb39ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:12.338486 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1af020-e46d-4e98-a447-7749d7fb39ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:12.338972 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1af020-e46d-4e98-a447-7749d7fb39ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:12.339178 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1af020-e46d-4e98-a447-7749d7fb39ca admin admin] PATH_INFO: `/v3/projects/b27fc770f9ce456ebfbf373a9836f3e9` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:12.340395 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1af020-e46d-4e98-a447-7749d7fb39ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=b27fc770f9ce456ebfbf373a9836f3e9)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:12.342387 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1af020-e46d-4e98-a447-7749d7fb39ca admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:12.396916 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-1c1af020-e46d-4e98-a447-7749d7fb39ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b27fc770f9ce456ebfbf373a9836f3e9', 'request_id': 'req-1c1af020-e46d-4e98-a447-7749d7fb39ca'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:12.404294 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-1c1af020-e46d-4e98-a447-7749d7fb39ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b27fc770f9ce456ebfbf373a9836f3e9', 'request_id': 'req-1c1af020-e46d-4e98-a447-7749d7fb39ca'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:12.410919 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-1c1af020-e46d-4e98-a447-7749d7fb39ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b27fc770f9ce456ebfbf373a9836f3e9', 'request_id': 'req-1c1af020-e46d-4e98-a447-7749d7fb39ca'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:12.426322 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-1c1af020-e46d-4e98-a447-7749d7fb39ca admin admin] The token cache is being invalidate because project b27fc770f9ce456ebfbf373a9836f3e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81618) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:49:12.427572 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-1c1af020-e46d-4e98-a447-7749d7fb39ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:12.428239 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-1c1af020-e46d-4e98-a447-7749d7fb39ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:12.429780 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1229/2465] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:49:12 2022] DELETE /identity/v3/projects/b27fc770f9ce456ebfbf373a9836f3e9 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:49:12.437543 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b52abb-0c4c-4951-a500-1c2494d1b330 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:12.437913 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b52abb-0c4c-4951-a500-1c2494d1b330 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:12.438266 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b52abb-0c4c-4951-a500-1c2494d1b330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:12.462440 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-06b52abb-0c4c-4951-a500-1c2494d1b330 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:49:12.463932 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-06b52abb-0c4c-4951-a500-1c2494d1b330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:12.548481 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1237/2466] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:49:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:49:12.556625 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a106ff6d-0cfd-4865-9a76-bda5c6a76161 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:12.557025 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a106ff6d-0cfd-4865-9a76-bda5c6a76161 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:12.557469 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a106ff6d-0cfd-4865-9a76-bda5c6a76161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:12.579018 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-a106ff6d-0cfd-4865-9a76-bda5c6a76161 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:49:12.580381 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-a106ff6d-0cfd-4865-9a76-bda5c6a76161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:12.590331 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1230/2467] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:49:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:49:12.594590 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b52abb-0c4c-4951-a500-1c2494d1b330 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:12.610539 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b52abb-0c4c-4951-a500-1c2494d1b330 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:12.615663 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5666fe-6008-4216-ae19-e7629aa97f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:12.616412 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5666fe-6008-4216-ae19-e7629aa97f42 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:12.617019 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5666fe-6008-4216-ae19-e7629aa97f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:12.617650 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5666fe-6008-4216-ae19-e7629aa97f42 admin admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:12.618736 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5666fe-6008-4216-ae19-e7629aa97f42 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:49:12.619246 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5666fe-6008-4216-ae19-e7629aa97f42 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:12.621583 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5666fe-6008-4216-ae19-e7629aa97f42 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:12.635979 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1238/2468] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:49:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:49:12.641408 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a106ff6d-0cfd-4865-9a76-bda5c6a76161 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:12.651162 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a106ff6d-0cfd-4865-9a76-bda5c6a76161 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:12.654990 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5666fe-6008-4216-ae19-e7629aa97f42 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:12.657194 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe10ec3b-49cf-4263-a83e-095ec27df2f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:12.657882 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe10ec3b-49cf-4263-a83e-095ec27df2f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:12.658026 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-0d5666fe-6008-4216-ae19-e7629aa97f42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:12.658336 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe10ec3b-49cf-4263-a83e-095ec27df2f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:12.658750 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe10ec3b-49cf-4263-a83e-095ec27df2f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:12.659482 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe10ec3b-49cf-4263-a83e-095ec27df2f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:12.661460 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe10ec3b-49cf-4263-a83e-095ec27df2f8 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:12.675113 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5666fe-6008-4216-ae19-e7629aa97f42 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:12.681637 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c42ec79-092f-4745-802b-0d9d264ec82b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:12.682346 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c42ec79-092f-4745-802b-0d9d264ec82b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:12.683252 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c42ec79-092f-4745-802b-0d9d264ec82b admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:12.683533 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c42ec79-092f-4745-802b-0d9d264ec82b admin admin] PATH_INFO: `/v3/projects/614b1aa8141140aea0d6af28d123d9d9` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:12.685763 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c42ec79-092f-4745-802b-0d9d264ec82b admin admin] RBAC: Authorizing `identity:delete_project(project_id=614b1aa8141140aea0d6af28d123d9d9)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:12.688226 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c42ec79-092f-4745-802b-0d9d264ec82b admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:12.712761 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1231/2469] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:49:12 2022] POST /identity/v3/projects => generated 367 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:49:12.720476 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe10ec3b-49cf-4263-a83e-095ec27df2f8 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:12.791882 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-9c42ec79-092f-4745-802b-0d9d264ec82b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '614b1aa8141140aea0d6af28d123d9d9', 'request_id': 'req-9c42ec79-092f-4745-802b-0d9d264ec82b'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:12.801131 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-9c42ec79-092f-4745-802b-0d9d264ec82b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '614b1aa8141140aea0d6af28d123d9d9', 'request_id': 'req-9c42ec79-092f-4745-802b-0d9d264ec82b'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:12.808263 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-9c42ec79-092f-4745-802b-0d9d264ec82b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '614b1aa8141140aea0d6af28d123d9d9', 'request_id': 'req-9c42ec79-092f-4745-802b-0d9d264ec82b'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:12.829914 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-9c42ec79-092f-4745-802b-0d9d264ec82b admin admin] The token cache is being invalidate because project 614b1aa8141140aea0d6af28d123d9d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81617) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:49:12.830204 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-9c42ec79-092f-4745-802b-0d9d264ec82b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:12.832697 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-9c42ec79-092f-4745-802b-0d9d264ec82b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:12.833881 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:DELETE=9,keystone:SELECT=88,keystone:INSERT=3 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:12.834897 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1239/2470] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:49:12 2022] DELETE /identity/v3/projects/614b1aa8141140aea0d6af28d123d9d9 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 16:49:12.841532 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae335829-f658-4cb1-b043-ff0d0bfa865c None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:12.841811 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae335829-f658-4cb1-b043-ff0d0bfa865c None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:12.842088 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae335829-f658-4cb1-b043-ff0d0bfa865c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:12.861922 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe10ec3b-49cf-4263-a83e-095ec27df2f8 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:12.878987 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-ae335829-f658-4cb1-b043-ff0d0bfa865c None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:49:12.880531 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-ae335829-f658-4cb1-b043-ff0d0bfa865c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:12.938293 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f394cbd0-9b06-45c3-82b0-14cb87a109b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:12.939235 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f394cbd0-9b06-45c3-82b0-14cb87a109b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:12.939688 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f394cbd0-9b06-45c3-82b0-14cb87a109b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:12.940455 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f394cbd0-9b06-45c3-82b0-14cb87a109b8 admin admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:12.941276 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f394cbd0-9b06-45c3-82b0-14cb87a109b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:12.943605 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f394cbd0-9b06-45c3-82b0-14cb87a109b8 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:12.968056 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1232/2471] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:49:12 2022] POST /identity/v3/users => generated 284 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:49:12.971298 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f394cbd0-9b06-45c3-82b0-14cb87a109b8 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:12.974249 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-f394cbd0-9b06-45c3-82b0-14cb87a109b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:12.987050 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f394cbd0-9b06-45c3-82b0-14cb87a109b8 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:13.000490 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1240/2472] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:49:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:49:13.001164 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5802e879-ef72-45ab-8757-d2a52b87dece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:13.001164 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5802e879-ef72-45ab-8757-d2a52b87dece admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:13.001164 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5802e879-ef72-45ab-8757-d2a52b87dece admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:13.001164 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5802e879-ef72-45ab-8757-d2a52b87dece admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:13.001770 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5802e879-ef72-45ab-8757-d2a52b87dece admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:13.001770 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5802e879-ef72-45ab-8757-d2a52b87dece admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:13.002245 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d24988-45ee-4bbf-9ffb-a8cb2393f51b None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:13.002573 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d24988-45ee-4bbf-9ffb-a8cb2393f51b None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:13.002850 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d24988-45ee-4bbf-9ffb-a8cb2393f51b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:13.019445 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1233/2473] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:49:12 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:49:13.019445 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5802e879-ef72-45ab-8757-d2a52b87dece admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:13.019932 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:INSERT=2,keystone:DELETE=8,keystone:SELECT=90 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:13.035075 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5802e879-ef72-45ab-8757-d2a52b87dece admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:13.036376 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-74d24988-45ee-4bbf-9ffb-a8cb2393f51b None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:49:13.040872 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-74d24988-45ee-4bbf-9ffb-a8cb2393f51b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:13.044236 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c2edf3-0271-41a3-a286-4fca5b77bf51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:13.044945 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c2edf3-0271-41a3-a286-4fca5b77bf51 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:13.045852 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c2edf3-0271-41a3-a286-4fca5b77bf51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:13.046432 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c2edf3-0271-41a3-a286-4fca5b77bf51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:13.048964 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c2edf3-0271-41a3-a286-4fca5b77bf51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:13.051856 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c2edf3-0271-41a3-a286-4fca5b77bf51 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:13.061573 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1241/2474] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:49:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:49:13.067006 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d24988-45ee-4bbf-9ffb-a8cb2393f51b None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:13.084866 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1234/2475] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:49:13 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:49:13.087482 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d24988-45ee-4bbf-9ffb-a8cb2393f51b None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:13.087692 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c2edf3-0271-41a3-a286-4fca5b77bf51 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:13.097224 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4c652c-dce6-468e-b432-5410ed68e222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:13.098359 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c652c-dce6-468e-b432-5410ed68e222 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:13.098960 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c652c-dce6-468e-b432-5410ed68e222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:13.099549 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c652c-dce6-468e-b432-5410ed68e222 admin admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:13.100949 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4c652c-dce6-468e-b432-5410ed68e222 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:49:13.101611 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4c652c-dce6-468e-b432-5410ed68e222 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:13.102127 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c2edf3-0271-41a3-a286-4fca5b77bf51 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:13.104947 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4c652c-dce6-468e-b432-5410ed68e222 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:13.110087 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1665b215-bf44-4941-b6e4-5f0da6d9b56a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:13.111185 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1665b215-bf44-4941-b6e4-5f0da6d9b56a admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:13.111632 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1665b215-bf44-4941-b6e4-5f0da6d9b56a admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:13.112212 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1665b215-bf44-4941-b6e4-5f0da6d9b56a admin admin] PATH_INFO: `/v3/projects/c15aca9ee17541a58651a57d2472af0b/users/b56de7294c1b46aaa38147ea1a995e39/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:13.124439 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1242/2476] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:49:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:49:13.128935 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4c652c-dce6-468e-b432-5410ed68e222 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:13.140415 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1665b215-bf44-4941-b6e4-5f0da6d9b56a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c15aca9ee17541a58651a57d2472af0b, user_id=b56de7294c1b46aaa38147ea1a995e39, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:13.142924 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4c652c-dce6-468e-b432-5410ed68e222 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:13.144316 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1665b215-bf44-4941-b6e4-5f0da6d9b56a admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:13.151652 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a02354-e3bf-4afb-9843-e8a26f86c6f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:13.152952 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a02354-e3bf-4afb-9843-e8a26f86c6f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:13.152952 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a02354-e3bf-4afb-9843-e8a26f86c6f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:13.152952 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a02354-e3bf-4afb-9843-e8a26f86c6f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:13.153679 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a02354-e3bf-4afb-9843-e8a26f86c6f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:13.156187 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a02354-e3bf-4afb-9843-e8a26f86c6f3 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:13.175065 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1235/2477] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:49:13 2022] PUT /identity/v3/projects/c15aca9ee17541a58651a57d2472af0b/users/b56de7294c1b46aaa38147ea1a995e39/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:49:13.176392 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d0a3ef-7cb9-4e38-a7cc-cc3436f072f6 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:13.176938 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d0a3ef-7cb9-4e38-a7cc-cc3436f072f6 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:13.177186 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d0a3ef-7cb9-4e38-a7cc-cc3436f072f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:13.199786 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1243/2478] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:49:13 2022] POST /identity/v3/projects => generated 373 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:49:13.203328 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a02354-e3bf-4afb-9843-e8a26f86c6f3 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:13.210969 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-43d0a3ef-7cb9-4e38-a7cc-cc3436f072f6 None None] MFA Rules not processed for user `b56de7294c1b46aaa38147ea1a995e39`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:49:13.212201 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-43d0a3ef-7cb9-4e38-a7cc-cc3436f072f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:13.278731 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1236/2479] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:49:13 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:49:13.283281 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d0a3ef-7cb9-4e38-a7cc-cc3436f072f6 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:13.283457 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a02354-e3bf-4afb-9843-e8a26f86c6f3 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:13.288134 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1106780-44df-4428-87d6-29557bc9ae5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:13.288726 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1106780-44df-4428-87d6-29557bc9ae5a admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:13.289080 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1106780-44df-4428-87d6-29557bc9ae5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:13.289367 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1106780-44df-4428-87d6-29557bc9ae5a admin admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:13.290015 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1106780-44df-4428-87d6-29557bc9ae5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:13.292101 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1106780-44df-4428-87d6-29557bc9ae5a admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:13.293811 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d0a3ef-7cb9-4e38-a7cc-cc3436f072f6 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:13.298523 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d214f7-ba1d-4f26-8528-7d025d548078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:13.299124 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d214f7-ba1d-4f26-8528-7d025d548078 admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:13.299405 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d214f7-ba1d-4f26-8528-7d025d548078 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:13.299677 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d214f7-ba1d-4f26-8528-7d025d548078 admin admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:13.300315 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d214f7-ba1d-4f26-8528-7d025d548078 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:13.302199 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d214f7-ba1d-4f26-8528-7d025d548078 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:13.316295 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1244/2480] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:49:13 2022] POST /identity/v3/users => generated 287 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 16:49:13.321055 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1106780-44df-4428-87d6-29557bc9ae5a admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:13.331904 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1106780-44df-4428-87d6-29557bc9ae5a admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:13.332750 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1237/2481] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:49:13 2022] POST /identity/v3/projects => generated 365 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 16:49:13.336987 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d214f7-ba1d-4f26-8528-7d025d548078 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:13.346010 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:13.401286 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2e0ca8-fda5-4513-ab14-262a02f56b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:13.402091 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2e0ca8-fda5-4513-ab14-262a02f56b51 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:13.402482 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2e0ca8-fda5-4513-ab14-262a02f56b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:13.402775 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2e0ca8-fda5-4513-ab14-262a02f56b51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:13.403596 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2e0ca8-fda5-4513-ab14-262a02f56b51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:13.405413 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2e0ca8-fda5-4513-ab14-262a02f56b51 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:13.413885 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d214f7-ba1d-4f26-8528-7d025d548078 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:13.417206 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1245/2482] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:49:13 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:49:13.420822 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383bb1e9-02b6-41d9-89b4-edd6c629e438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:13.421474 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383bb1e9-02b6-41d9-89b4-edd6c629e438 admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:13.421755 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383bb1e9-02b6-41d9-89b4-edd6c629e438 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:13.422027 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383bb1e9-02b6-41d9-89b4-edd6c629e438 admin admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:13.422707 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383bb1e9-02b6-41d9-89b4-edd6c629e438 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:13.423484 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2e0ca8-fda5-4513-ab14-262a02f56b51 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:13.424863 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383bb1e9-02b6-41d9-89b4-edd6c629e438 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:13.442809 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2e0ca8-fda5-4513-ab14-262a02f56b51 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:13.449519 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01950e71-d735-4e0f-9c2e-1bab41bca3ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:13.450277 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01950e71-d735-4e0f-9c2e-1bab41bca3ee admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:13.451034 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01950e71-d735-4e0f-9c2e-1bab41bca3ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:13.451238 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1238/2483] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:49:13 2022] POST /identity/v3/users => generated 289 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 16:49:13.454077 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01950e71-d735-4e0f-9c2e-1bab41bca3ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:13.454752 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383bb1e9-02b6-41d9-89b4-edd6c629e438 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:13.455200 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01950e71-d735-4e0f-9c2e-1bab41bca3ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:13.456788 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01950e71-d735-4e0f-9c2e-1bab41bca3ee admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:13.465408 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383bb1e9-02b6-41d9-89b4-edd6c629e438 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:13.468122 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1246/2484] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:49:13 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:49:13.470929 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b99a47d-8d16-43bb-bf51-b4b08947dee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:13.471615 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b99a47d-8d16-43bb-bf51-b4b08947dee8 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:13.471946 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b99a47d-8d16-43bb-bf51-b4b08947dee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:13.472227 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b99a47d-8d16-43bb-bf51-b4b08947dee8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:13.472494 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01950e71-d735-4e0f-9c2e-1bab41bca3ee admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:13.473016 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b99a47d-8d16-43bb-bf51-b4b08947dee8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:13.474827 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b99a47d-8d16-43bb-bf51-b4b08947dee8 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:13.483137 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01950e71-d735-4e0f-9c2e-1bab41bca3ee admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:13.486734 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1239/2485] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:49:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:49:13.490759 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b99a47d-8d16-43bb-bf51-b4b08947dee8 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:13.492119 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d542c85-a598-44df-9139-81b6fefe0f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:13.492857 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d542c85-a598-44df-9139-81b6fefe0f37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:13.493360 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d542c85-a598-44df-9139-81b6fefe0f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:13.493561 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d542c85-a598-44df-9139-81b6fefe0f37 admin admin] PATH_INFO: `/v3/projects/2199201c2a674266894b5c5caba834e7/users/3ced71188e0a4e4184ea997239965374/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:13.502480 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b99a47d-8d16-43bb-bf51-b4b08947dee8 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:13.507578 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4ee35e-b47c-43dd-9bc2-9828479641b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:13.508507 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4ee35e-b47c-43dd-9bc2-9828479641b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:13.508507 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4ee35e-b47c-43dd-9bc2-9828479641b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:13.508785 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4ee35e-b47c-43dd-9bc2-9828479641b0 admin admin] PATH_INFO: `/v3/projects/79a46b99b1b04cf29fc19a2d8aa827cd/users/9d5167905fe043b8b5c58cd783b9badc/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:13.513418 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d542c85-a598-44df-9139-81b6fefe0f37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2199201c2a674266894b5c5caba834e7, user_id=3ced71188e0a4e4184ea997239965374, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:13.517339 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d542c85-a598-44df-9139-81b6fefe0f37 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:13.524850 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4ee35e-b47c-43dd-9bc2-9828479641b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79a46b99b1b04cf29fc19a2d8aa827cd, user_id=9d5167905fe043b8b5c58cd783b9badc, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:13.527670 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4ee35e-b47c-43dd-9bc2-9828479641b0 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:13.536950 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1247/2486] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:49:13 2022] PUT /identity/v3/projects/2199201c2a674266894b5c5caba834e7/users/3ced71188e0a4e4184ea997239965374/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 16:49:13.546260 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1240/2487] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:49:13 2022] PUT /identity/v3/projects/79a46b99b1b04cf29fc19a2d8aa827cd/users/9d5167905fe043b8b5c58cd783b9badc/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 16:49:13.548104 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d542c85-a598-44df-9139-81b6fefe0f37 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:13.548821 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c47cbf5-a093-44f7-bc0e-c8894377b42d None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:13.549072 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c47cbf5-a093-44f7-bc0e-c8894377b42d None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:13.549416 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c47cbf5-a093-44f7-bc0e-c8894377b42d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:13.563766 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:13.582259 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-8c47cbf5-a093-44f7-bc0e-c8894377b42d None None] MFA Rules not processed for user `3ced71188e0a4e4184ea997239965374`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:49:13.583508 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-8c47cbf5-a093-44f7-bc0e-c8894377b42d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:13.636422 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d542c85-a598-44df-9139-81b6fefe0f37 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:13.705755 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5944f6c6-1d89-410d-afb4-9b86c2ecdd7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:13.706539 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944f6c6-1d89-410d-afb4-9b86c2ecdd7e admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:13.706889 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944f6c6-1d89-410d-afb4-9b86c2ecdd7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:13.707238 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944f6c6-1d89-410d-afb4-9b86c2ecdd7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:13.707960 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5944f6c6-1d89-410d-afb4-9b86c2ecdd7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:13.709672 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5944f6c6-1d89-410d-afb4-9b86c2ecdd7e admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:13.721850 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1248/2488] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:49:13 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:49:13.726974 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5944f6c6-1d89-410d-afb4-9b86c2ecdd7e admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:13.738191 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5944f6c6-1d89-410d-afb4-9b86c2ecdd7e admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:13.744906 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec983ce-6284-45ee-ab1f-1f2938827100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:13.746930 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec983ce-6284-45ee-ab1f-1f2938827100 admin admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:13.747224 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec983ce-6284-45ee-ab1f-1f2938827100 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:13.747498 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec983ce-6284-45ee-ab1f-1f2938827100 admin admin] PATH_INFO: `/v3/projects/79a46b99b1b04cf29fc19a2d8aa827cd/users/9d5167905fe043b8b5c58cd783b9badc/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:13.749475 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec983ce-6284-45ee-ab1f-1f2938827100 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79a46b99b1b04cf29fc19a2d8aa827cd, user_id=9d5167905fe043b8b5c58cd783b9badc, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:13.752735 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec983ce-6284-45ee-ab1f-1f2938827100 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:13.770488 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1249/2489] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:49:13 2022] PUT /identity/v3/projects/79a46b99b1b04cf29fc19a2d8aa827cd/users/9d5167905fe043b8b5c58cd783b9badc/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 16:49:13.778226 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dd5e96-5f79-4fb8-97c7-29c765a4a452 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:13.778598 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dd5e96-5f79-4fb8-97c7-29c765a4a452 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:13.778943 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dd5e96-5f79-4fb8-97c7-29c765a4a452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:13.797515 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1241/2490] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:49:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:49:13.810722 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c47cbf5-a093-44f7-bc0e-c8894377b42d None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:13.813238 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-8c47cbf5-a093-44f7-bc0e-c8894377b42d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:13.822591 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-c1dd5e96-5f79-4fb8-97c7-29c765a4a452 None None] MFA Rules not processed for user `9d5167905fe043b8b5c58cd783b9badc`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:49:13.824638 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-c1dd5e96-5f79-4fb8-97c7-29c765a4a452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:13.887189 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c47cbf5-a093-44f7-bc0e-c8894377b42d None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:13.891216 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:13.933261 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1250/2491] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:49:13 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:49:13.948947 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dd5e96-5f79-4fb8-97c7-29c765a4a452 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:13.951113 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-c1dd5e96-5f79-4fb8-97c7-29c765a4a452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:13.962900 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2d2658-6b71-4915-bf2d-8e152cff6223 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:13.963864 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d2658-6b71-4915-bf2d-8e152cff6223 service cinder] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:13.964226 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d2658-6b71-4915-bf2d-8e152cff6223 service cinder] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:13.965327 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d2658-6b71-4915-bf2d-8e152cff6223 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:13.998103 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:14.043402 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dd5e96-5f79-4fb8-97c7-29c765a4a452 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:14.044800 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6708a30c-b5b0-403b-8168-a0f7d7a1843f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:14.045677 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708a30c-b5b0-403b-8168-a0f7d7a1843f service cinder] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:14.046530 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708a30c-b5b0-403b-8168-a0f7d7a1843f service cinder] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:14.047620 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708a30c-b5b0-403b-8168-a0f7d7a1843f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:14.050632 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2d2658-6b71-4915-bf2d-8e152cff6223 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:14.052069 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2d2658-6b71-4915-bf2d-8e152cff6223 service cinder] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:14.059903 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6708a30c-b5b0-403b-8168-a0f7d7a1843f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:14.061567 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6708a30c-b5b0-403b-8168-a0f7d7a1843f service cinder] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:14.063535 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1242/2492] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:49:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 16:49:14.086745 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1251/2493] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:49:13 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 16:49:14.089196 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2d2658-6b71-4915-bf2d-8e152cff6223 service cinder] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:14.183314 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2d2658-6b71-4915-bf2d-8e152cff6223 service cinder] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:14.189457 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ce7fde-827c-4636-b68b-310a9d84a0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:14.190293 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ce7fde-827c-4636-b68b-310a9d84a0f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:14.190607 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ce7fde-827c-4636-b68b-310a9d84a0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:14.190885 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ce7fde-827c-4636-b68b-310a9d84a0f5 admin admin] PATH_INFO: `/v3/users/3ced71188e0a4e4184ea997239965374` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:14.192068 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ce7fde-827c-4636-b68b-310a9d84a0f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ced71188e0a4e4184ea997239965374)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:14.194522 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ce7fde-827c-4636-b68b-310a9d84a0f5 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:14.242140 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6708a30c-b5b0-403b-8168-a0f7d7a1843f service cinder] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:14.267500 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6708a30c-b5b0-403b-8168-a0f7d7a1843f service cinder] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:14.279224 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d6b38-d970-46f9-8942-18c4478cc05d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:14.280183 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d6b38-d970-46f9-8942-18c4478cc05d service cinder] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:14.280636 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d6b38-d970-46f9-8942-18c4478cc05d service cinder] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:14.281021 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d6b38-d970-46f9-8942-18c4478cc05d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:14.314639 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-59ce7fde-827c-4636-b68b-310a9d84a0f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ced71188e0a4e4184ea997239965374'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:14.338762 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-59ce7fde-827c-4636-b68b-310a9d84a0f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ced71188e0a4e4184ea997239965374'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:14.354671 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:14.371146 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-59ce7fde-827c-4636-b68b-310a9d84a0f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ced71188e0a4e4184ea997239965374'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:14.383753 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-59ce7fde-827c-4636-b68b-310a9d84a0f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ced71188e0a4e4184ea997239965374'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:14.384515 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-59ce7fde-827c-4636-b68b-310a9d84a0f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ced71188e0a4e4184ea997239965374'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:14.402225 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1243/2494] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:49:14 2022] DELETE /identity/v3/users/3ced71188e0a4e4184ea997239965374 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:49:14.408165 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86891de8-b0a5-4811-8e51-8b9077235ab2 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:14.408631 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86891de8-b0a5-4811-8e51-8b9077235ab2 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:14.408998 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86891de8-b0a5-4811-8e51-8b9077235ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:14.434210 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-86891de8-b0a5-4811-8e51-8b9077235ab2 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:49:14.435909 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8d6b38-d970-46f9-8942-18c4478cc05d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:14.436079 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-86891de8-b0a5-4811-8e51-8b9077235ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:14.438052 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8d6b38-d970-46f9-8942-18c4478cc05d service cinder] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:14.459845 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1252/2495] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:49:14 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:49:14.583561 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1244/2496] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:49:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:49:14.597122 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d6b38-d970-46f9-8942-18c4478cc05d service cinder] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:14.601528 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-8b8d6b38-d970-46f9-8942-18c4478cc05d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:14.672404 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d6b38-d970-46f9-8942-18c4478cc05d service cinder] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:14.741224 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f244856-2616-4b85-a687-277124c40fc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:14.742229 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f244856-2616-4b85-a687-277124c40fc6 service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:14.742684 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f244856-2616-4b85-a687-277124c40fc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:14.742971 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f244856-2616-4b85-a687-277124c40fc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:14.753546 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f244856-2616-4b85-a687-277124c40fc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:14.755036 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f244856-2616-4b85-a687-277124c40fc6 service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:14.770616 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1253/2497] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:49:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:49:15.076751 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86891de8-b0a5-4811-8e51-8b9077235ab2 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:15.078775 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-86891de8-b0a5-4811-8e51-8b9077235ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:15.090869 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86891de8-b0a5-4811-8e51-8b9077235ab2 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:15.095809 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd48ac1-26cb-4cc9-9e07-f182d5f6d2c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:15.096949 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd48ac1-26cb-4cc9-9e07-f182d5f6d2c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:15.096949 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd48ac1-26cb-4cc9-9e07-f182d5f6d2c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:15.096949 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd48ac1-26cb-4cc9-9e07-f182d5f6d2c5 admin admin] PATH_INFO: `/v3/projects/2199201c2a674266894b5c5caba834e7` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:15.098105 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd48ac1-26cb-4cc9-9e07-f182d5f6d2c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2199201c2a674266894b5c5caba834e7)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:15.100206 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd48ac1-26cb-4cc9-9e07-f182d5f6d2c5 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:15.151136 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-5dd48ac1-26cb-4cc9-9e07-f182d5f6d2c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2199201c2a674266894b5c5caba834e7', 'request_id': 'req-5dd48ac1-26cb-4cc9-9e07-f182d5f6d2c5'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:15.157228 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-5dd48ac1-26cb-4cc9-9e07-f182d5f6d2c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2199201c2a674266894b5c5caba834e7', 'request_id': 'req-5dd48ac1-26cb-4cc9-9e07-f182d5f6d2c5'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:15.162635 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-5dd48ac1-26cb-4cc9-9e07-f182d5f6d2c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2199201c2a674266894b5c5caba834e7', 'request_id': 'req-5dd48ac1-26cb-4cc9-9e07-f182d5f6d2c5'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:15.174831 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-5dd48ac1-26cb-4cc9-9e07-f182d5f6d2c5 admin admin] The token cache is being invalidate because project 2199201c2a674266894b5c5caba834e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81618) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:49:15.175115 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-5dd48ac1-26cb-4cc9-9e07-f182d5f6d2c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:15.175722 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-5dd48ac1-26cb-4cc9-9e07-f182d5f6d2c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:15.177172 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1245/2498] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:49:15 2022] DELETE /identity/v3/projects/2199201c2a674266894b5c5caba834e7 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:49:15.183023 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0fabde-9ccc-4f81-bffe-e440687251da None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:15.183312 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0fabde-9ccc-4f81-bffe-e440687251da None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:15.183591 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0fabde-9ccc-4f81-bffe-e440687251da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:15.202999 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-3a0fabde-9ccc-4f81-bffe-e440687251da None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:49:15.204197 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-3a0fabde-9ccc-4f81-bffe-e440687251da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:15.224062 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:15.263235 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1254/2499] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:49:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:49:15.270977 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f65fb-92d5-436f-9571-edc20ccb8e6d None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:15.271263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f65fb-92d5-436f-9571-edc20ccb8e6d None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:15.271540 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f65fb-92d5-436f-9571-edc20ccb8e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:15.291853 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-f11f65fb-92d5-436f-9571-edc20ccb8e6d None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:49:15.293175 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-f11f65fb-92d5-436f-9571-edc20ccb8e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:15.300950 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1246/2500] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:49:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:49:15.305151 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0fabde-9ccc-4f81-bffe-e440687251da None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:15.315309 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0fabde-9ccc-4f81-bffe-e440687251da None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:15.320051 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e668944a-66f6-45de-9981-acd4e42cd9f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:15.320603 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e668944a-66f6-45de-9981-acd4e42cd9f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:15.320887 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e668944a-66f6-45de-9981-acd4e42cd9f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:15.321164 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e668944a-66f6-45de-9981-acd4e42cd9f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:15.321816 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e668944a-66f6-45de-9981-acd4e42cd9f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:49:15.322121 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e668944a-66f6-45de-9981-acd4e42cd9f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:15.323758 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e668944a-66f6-45de-9981-acd4e42cd9f6 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:15.336353 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1255/2501] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:49:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:49:15.340670 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11f65fb-92d5-436f-9571-edc20ccb8e6d None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:15.349728 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11f65fb-92d5-436f-9571-edc20ccb8e6d None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:15.355536 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a02dbb8-5c23-4088-ba3c-1878af91d131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:15.356128 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a02dbb8-5c23-4088-ba3c-1878af91d131 admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:15.356433 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a02dbb8-5c23-4088-ba3c-1878af91d131 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:15.356699 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a02dbb8-5c23-4088-ba3c-1878af91d131 admin admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:15.357323 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a02dbb8-5c23-4088-ba3c-1878af91d131 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:15.359217 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a02dbb8-5c23-4088-ba3c-1878af91d131 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:15.386556 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1247/2502] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:49:15 2022] POST /identity/v3/projects => generated 359 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:49:15.390809 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e668944a-66f6-45de-9981-acd4e42cd9f6 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:15.401603 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e668944a-66f6-45de-9981-acd4e42cd9f6 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:15.457327 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49a1896-41b4-4931-8141-80b6a0a5dc0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:15.457958 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49a1896-41b4-4931-8141-80b6a0a5dc0d admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:15.458241 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49a1896-41b4-4931-8141-80b6a0a5dc0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:15.458562 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49a1896-41b4-4931-8141-80b6a0a5dc0d admin admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:15.459238 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49a1896-41b4-4931-8141-80b6a0a5dc0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:15.461631 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49a1896-41b4-4931-8141-80b6a0a5dc0d admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:15.483470 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1256/2503] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:49:15 2022] POST /identity/v3/users => generated 280 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:49:15.487603 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a02dbb8-5c23-4088-ba3c-1878af91d131 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:15.498079 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a02dbb8-5c23-4088-ba3c-1878af91d131 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:15.502773 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d555be-e4ee-45c5-9b19-3e0e33e43c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:15.503404 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d555be-e4ee-45c5-9b19-3e0e33e43c1d admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:15.503682 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d555be-e4ee-45c5-9b19-3e0e33e43c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:15.503950 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d555be-e4ee-45c5-9b19-3e0e33e43c1d admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:15.504611 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d555be-e4ee-45c5-9b19-3e0e33e43c1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:15.506195 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d555be-e4ee-45c5-9b19-3e0e33e43c1d admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:15.516005 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1248/2504] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:49:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:49:15.519795 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49a1896-41b4-4931-8141-80b6a0a5dc0d admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:15.530232 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49a1896-41b4-4931-8141-80b6a0a5dc0d admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:15.534896 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada61c5b-b89b-4106-b1cd-aa2ee98c6cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:15.535487 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada61c5b-b89b-4106-b1cd-aa2ee98c6cf1 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:15.535781 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada61c5b-b89b-4106-b1cd-aa2ee98c6cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:15.536044 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada61c5b-b89b-4106-b1cd-aa2ee98c6cf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:15.536683 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada61c5b-b89b-4106-b1cd-aa2ee98c6cf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:15.539123 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada61c5b-b89b-4106-b1cd-aa2ee98c6cf1 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:15.549196 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1257/2505] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:49:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:49:15.553235 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d555be-e4ee-45c5-9b19-3e0e33e43c1d admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:15.563337 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d555be-e4ee-45c5-9b19-3e0e33e43c1d admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:15.567858 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe87a3ec-305a-4492-98dc-90488baee1c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:15.568496 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe87a3ec-305a-4492-98dc-90488baee1c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:15.568771 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe87a3ec-305a-4492-98dc-90488baee1c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:15.569040 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe87a3ec-305a-4492-98dc-90488baee1c1 admin admin] PATH_INFO: `/v3/projects/6013953834054ccea3bda143fd6a79ef/users/e15fa7baa8064297ad900663c6089c6e/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:15.583413 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe87a3ec-305a-4492-98dc-90488baee1c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6013953834054ccea3bda143fd6a79ef, user_id=e15fa7baa8064297ad900663c6089c6e, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:15.586509 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe87a3ec-305a-4492-98dc-90488baee1c1 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:15.595384 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:15.602480 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1249/2506] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:49:15 2022] PUT /identity/v3/projects/6013953834054ccea3bda143fd6a79ef/users/e15fa7baa8064297ad900663c6089c6e/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:49:15.617783 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1904c8aa-1808-4476-bd16-1cebd3a6fe48 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:15.618069 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1904c8aa-1808-4476-bd16-1cebd3a6fe48 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:15.618350 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1904c8aa-1808-4476-bd16-1cebd3a6fe48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:15.653138 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-1904c8aa-1808-4476-bd16-1cebd3a6fe48 None None] MFA Rules not processed for user `e15fa7baa8064297ad900663c6089c6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:49:15.654096 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-1904c8aa-1808-4476-bd16-1cebd3a6fe48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:15.719271 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1258/2507] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:49:15 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:49:15.754543 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe87a3ec-305a-4492-98dc-90488baee1c1 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:15.756321 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-fe87a3ec-305a-4492-98dc-90488baee1c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:15.841881 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe87a3ec-305a-4492-98dc-90488baee1c1 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:15.845783 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbeb975-1051-47ea-a92d-e045993ed439 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:15.846408 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbeb975-1051-47ea-a92d-e045993ed439 service cinder] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:15.846694 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbeb975-1051-47ea-a92d-e045993ed439 service cinder] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:15.846968 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbeb975-1051-47ea-a92d-e045993ed439 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:15.855759 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbeb975-1051-47ea-a92d-e045993ed439 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:15.857188 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbeb975-1051-47ea-a92d-e045993ed439 service cinder] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:15.866838 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1250/2508] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:49:15 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:49:15.884008 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1904c8aa-1808-4476-bd16-1cebd3a6fe48 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:15.911851 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:15.953730 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1904c8aa-1808-4476-bd16-1cebd3a6fe48 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:15.959596 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684f699e-4c7f-4fb1-aeae-eb103acce169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:15.961220 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f699e-4c7f-4fb1-aeae-eb103acce169 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:15.961683 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f699e-4c7f-4fb1-aeae-eb103acce169 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:15.962216 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f699e-4c7f-4fb1-aeae-eb103acce169 admin admin] PATH_INFO: `/v3/users/e15fa7baa8064297ad900663c6089c6e` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:15.963451 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684f699e-4c7f-4fb1-aeae-eb103acce169 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e15fa7baa8064297ad900663c6089c6e)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:15.965433 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684f699e-4c7f-4fb1-aeae-eb103acce169 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:16.018131 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-684f699e-4c7f-4fb1-aeae-eb103acce169 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e15fa7baa8064297ad900663c6089c6e'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:16.018803 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-684f699e-4c7f-4fb1-aeae-eb103acce169 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e15fa7baa8064297ad900663c6089c6e'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:16.026158 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-684f699e-4c7f-4fb1-aeae-eb103acce169 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e15fa7baa8064297ad900663c6089c6e'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:16.037415 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-684f699e-4c7f-4fb1-aeae-eb103acce169 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e15fa7baa8064297ad900663c6089c6e'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:16.038094 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-684f699e-4c7f-4fb1-aeae-eb103acce169 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e15fa7baa8064297ad900663c6089c6e'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:16.050455 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1259/2509] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:49:15 2022] DELETE /identity/v3/users/e15fa7baa8064297ad900663c6089c6e => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:49:16.055418 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e794182a-3f28-4cb8-a77c-17cb3c461c0e None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:16.055732 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e794182a-3f28-4cb8-a77c-17cb3c461c0e None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:16.056004 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e794182a-3f28-4cb8-a77c-17cb3c461c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:16.075462 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-e794182a-3f28-4cb8-a77c-17cb3c461c0e None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:49:16.076654 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-e794182a-3f28-4cb8-a77c-17cb3c461c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:16.135842 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1251/2510] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:49:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:49:16.146997 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684f699e-4c7f-4fb1-aeae-eb103acce169 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:16.149453 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-684f699e-4c7f-4fb1-aeae-eb103acce169 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:16.211884 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684f699e-4c7f-4fb1-aeae-eb103acce169 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:16.215694 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4737fc5a-2410-4b09-a333-babbaf5735a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:16.216304 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4737fc5a-2410-4b09-a333-babbaf5735a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:16.216584 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4737fc5a-2410-4b09-a333-babbaf5735a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:16.216856 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4737fc5a-2410-4b09-a333-babbaf5735a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:16.226285 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4737fc5a-2410-4b09-a333-babbaf5735a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:16.227742 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4737fc5a-2410-4b09-a333-babbaf5735a8 service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:16.239539 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1260/2511] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:49:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:49:16.513445 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e794182a-3f28-4cb8-a77c-17cb3c461c0e None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:16.515121 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-e794182a-3f28-4cb8-a77c-17cb3c461c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:16.532202 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e794182a-3f28-4cb8-a77c-17cb3c461c0e None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:16.537713 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e695dc-2462-4d01-bcab-cff90816daeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:16.539648 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e695dc-2462-4d01-bcab-cff90816daeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:16.539934 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e695dc-2462-4d01-bcab-cff90816daeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:16.540183 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e695dc-2462-4d01-bcab-cff90816daeb admin admin] PATH_INFO: `/v3/projects/6013953834054ccea3bda143fd6a79ef` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:16.541519 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e695dc-2462-4d01-bcab-cff90816daeb admin admin] RBAC: Authorizing `identity:delete_project(project_id=6013953834054ccea3bda143fd6a79ef)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:16.543886 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e695dc-2462-4d01-bcab-cff90816daeb admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:16.591320 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-d9e695dc-2462-4d01-bcab-cff90816daeb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6013953834054ccea3bda143fd6a79ef', 'request_id': 'req-d9e695dc-2462-4d01-bcab-cff90816daeb'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:16.598123 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-d9e695dc-2462-4d01-bcab-cff90816daeb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6013953834054ccea3bda143fd6a79ef', 'request_id': 'req-d9e695dc-2462-4d01-bcab-cff90816daeb'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:16.604855 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-d9e695dc-2462-4d01-bcab-cff90816daeb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6013953834054ccea3bda143fd6a79ef', 'request_id': 'req-d9e695dc-2462-4d01-bcab-cff90816daeb'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:16.618482 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-d9e695dc-2462-4d01-bcab-cff90816daeb admin admin] The token cache is being invalidate because project 6013953834054ccea3bda143fd6a79ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81618) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:49:16.618840 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-d9e695dc-2462-4d01-bcab-cff90816daeb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:16.619531 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-d9e695dc-2462-4d01-bcab-cff90816daeb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:16.621164 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1252/2512] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:49:16 2022] DELETE /identity/v3/projects/6013953834054ccea3bda143fd6a79ef => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:49:17.989010 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c683ac-19d8-48c1-99ed-187646a32875 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:17.989565 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c683ac-19d8-48c1-99ed-187646a32875 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:17.989565 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c683ac-19d8-48c1-99ed-187646a32875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:18.026052 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-d9c683ac-19d8-48c1-99ed-187646a32875 None None] MFA Rules not processed for user `e1c0b2cfaa9f485f9116e012c62120fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:49:18.027408 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-d9c683ac-19d8-48c1-99ed-187646a32875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:18.072200 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:18.119796 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1261/2513] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:49:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:49:18.504147 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e695dc-2462-4d01-bcab-cff90816daeb admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:18.505848 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-d9e695dc-2462-4d01-bcab-cff90816daeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:18.576955 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:18.630703 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e695dc-2462-4d01-bcab-cff90816daeb admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:18.637517 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097f0f32-a1d7-4368-b436-8a613ae10699 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:18.638629 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097f0f32-a1d7-4368-b436-8a613ae10699 service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:18.639139 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097f0f32-a1d7-4368-b436-8a613ae10699 service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:18.639600 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097f0f32-a1d7-4368-b436-8a613ae10699 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:18.642086 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-097f0f32-a1d7-4368-b436-8a613ae10699 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:18.658221 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097f0f32-a1d7-4368-b436-8a613ae10699 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:18.659958 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097f0f32-a1d7-4368-b436-8a613ae10699 service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:18.669861 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1253/2514] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:49:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:49:19.009468 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c683ac-19d8-48c1-99ed-187646a32875 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:19.010958 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-d9c683ac-19d8-48c1-99ed-187646a32875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:19.100111 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c683ac-19d8-48c1-99ed-187646a32875 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:19.105966 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b78bd-d885-4265-ba0e-2c29593e144d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1a985495ed48a4acc48e2e6fc7f977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:19.106781 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b78bd-d885-4265-ba0e-2c29593e144d service placement] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:19.107103 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b78bd-d885-4265-ba0e-2c29593e144d service placement] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:19.107394 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b78bd-d885-4265-ba0e-2c29593e144d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:19.109632 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-ae8b78bd-d885-4265-ba0e-2c29593e144d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:19.122923 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8b78bd-d885-4265-ba0e-2c29593e144d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:19.124658 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8b78bd-d885-4265-ba0e-2c29593e144d service placement] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:19.137525 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1262/2515] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:49:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:49:19.694161 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9b7d6-611a-4887-9996-ea2cb04e2e3b None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:19.695036 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9b7d6-611a-4887-9996-ea2cb04e2e3b None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:19.695036 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9b7d6-611a-4887-9996-ea2cb04e2e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:19.783239 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-9bc9b7d6-611a-4887-9996-ea2cb04e2e3b None None] MFA Rules not processed for user `7222bd3b313b47e5a4fa0ebc9ff0cdd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:49:19.784663 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-9bc9b7d6-611a-4887-9996-ea2cb04e2e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:19.870576 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1254/2516] 10.222.0.57 () {58 vars in 967 bytes} [Mon Sep 19 16:49:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:49:19.938171 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b78bd-d885-4265-ba0e-2c29593e144d service placement] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:19.939754 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-ae8b78bd-d885-4265-ba0e-2c29593e144d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:20.067808 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b78bd-d885-4265-ba0e-2c29593e144d service placement] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:20.076486 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343fa196-96af-4b07-94aa-7111cf575d27 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '884a0524aca744b99c7496c11c3291ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:20.076806 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343fa196-96af-4b07-94aa-7111cf575d27 service swift] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:20.077158 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343fa196-96af-4b07-94aa-7111cf575d27 service swift] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:20.077495 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343fa196-96af-4b07-94aa-7111cf575d27 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:20.095747 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343fa196-96af-4b07-94aa-7111cf575d27 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:20.098589 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343fa196-96af-4b07-94aa-7111cf575d27 service swift] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:20.112598 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1263/2517] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Sep 19 16:49:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 16:49:20.769168 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc9b7d6-611a-4887-9996-ea2cb04e2e3b None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:20.775003 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-9bc9b7d6-611a-4887-9996-ea2cb04e2e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:20.809899 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc9b7d6-611a-4887-9996-ea2cb04e2e3b None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:20.814581 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74fc4b1-77d3-4572-b6f6-02be486515b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:20.815431 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74fc4b1-77d3-4572-b6f6-02be486515b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:20.816052 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74fc4b1-77d3-4572-b6f6-02be486515b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:20.817305 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74fc4b1-77d3-4572-b6f6-02be486515b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:20.819835 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-e74fc4b1-77d3-4572-b6f6-02be486515b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:20.935259 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74fc4b1-77d3-4572-b6f6-02be486515b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:20.937412 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74fc4b1-77d3-4572-b6f6-02be486515b5 service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:20.969532 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1255/2518] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:49:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:49:20.975023 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260ed792-2dc5-4fe2-996e-34a751f41f4c None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:20.975527 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260ed792-2dc5-4fe2-996e-34a751f41f4c None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:20.975968 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260ed792-2dc5-4fe2-996e-34a751f41f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:21.006958 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-260ed792-2dc5-4fe2-996e-34a751f41f4c None None] MFA Rules not processed for user `7222bd3b313b47e5a4fa0ebc9ff0cdd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:49:21.008910 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-260ed792-2dc5-4fe2-996e-34a751f41f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:21.047107 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1264/2519] 10.222.0.57 () {58 vars in 1009 bytes} [Mon Sep 19 16:49:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:49:21.047345 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74fc4b1-77d3-4572-b6f6-02be486515b5 service neutron] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:21.073962 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74fc4b1-77d3-4572-b6f6-02be486515b5 service neutron] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:21.080204 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8894b7b-09ea-45dd-a7a0-33330106259f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '884a0524aca744b99c7496c11c3291ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:21.082342 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8894b7b-09ea-45dd-a7a0-33330106259f service swift] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:21.082786 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8894b7b-09ea-45dd-a7a0-33330106259f service swift] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:21.083449 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8894b7b-09ea-45dd-a7a0-33330106259f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:21.100731 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8894b7b-09ea-45dd-a7a0-33330106259f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:21.103085 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8894b7b-09ea-45dd-a7a0-33330106259f service swift] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:21.123008 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1256/2520] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Sep 19 16:49:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 16:49:22.620076 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260ed792-2dc5-4fe2-996e-34a751f41f4c None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:22.621770 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-260ed792-2dc5-4fe2-996e-34a751f41f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:22.641272 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8894b7b-09ea-45dd-a7a0-33330106259f service swift] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:22.643189 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-d8894b7b-09ea-45dd-a7a0-33330106259f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:22.658075 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260ed792-2dc5-4fe2-996e-34a751f41f4c None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:22.667879 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:22.676628 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c22e25-9c0a-47cb-b896-cd8522ce5476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:22.677569 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c22e25-9c0a-47cb-b896-cd8522ce5476 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:22.678593 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c22e25-9c0a-47cb-b896-cd8522ce5476 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:22.680651 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c22e25-9c0a-47cb-b896-cd8522ce5476 admin admin] PATH_INFO: `/v3/users/b56de7294c1b46aaa38147ea1a995e39` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:22.692606 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c22e25-9c0a-47cb-b896-cd8522ce5476 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b56de7294c1b46aaa38147ea1a995e39)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:22.696616 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c22e25-9c0a-47cb-b896-cd8522ce5476 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:22.787911 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:22.804885 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8894b7b-09ea-45dd-a7a0-33330106259f service swift] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:22.811760 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-d6c22e25-9c0a-47cb-b896-cd8522ce5476 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b56de7294c1b46aaa38147ea1a995e39'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:22.813080 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-d6c22e25-9c0a-47cb-b896-cd8522ce5476 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b56de7294c1b46aaa38147ea1a995e39'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:22.834343 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510abfee-a5a1-49cc-8c14-6705f663046a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:22.837018 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510abfee-a5a1-49cc-8c14-6705f663046a service nova] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:22.839195 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510abfee-a5a1-49cc-8c14-6705f663046a service nova] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:22.839738 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510abfee-a5a1-49cc-8c14-6705f663046a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:22.841065 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-d6c22e25-9c0a-47cb-b896-cd8522ce5476 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b56de7294c1b46aaa38147ea1a995e39'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:22.844052 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-510abfee-a5a1-49cc-8c14-6705f663046a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:22.874059 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510abfee-a5a1-49cc-8c14-6705f663046a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:22.876295 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510abfee-a5a1-49cc-8c14-6705f663046a service nova] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:22.877436 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-d6c22e25-9c0a-47cb-b896-cd8522ce5476 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b56de7294c1b46aaa38147ea1a995e39'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:22.884413 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-d6c22e25-9c0a-47cb-b896-cd8522ce5476 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b56de7294c1b46aaa38147ea1a995e39'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:22.916032 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1257/2521] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:49:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:49:22.931055 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1265/2522] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:49:22 2022] DELETE /identity/v3/users/b56de7294c1b46aaa38147ea1a995e39 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 16:49:22.937288 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919688d0-aa7c-469f-b0aa-71e31bc69798 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:22.937996 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919688d0-aa7c-469f-b0aa-71e31bc69798 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:22.938480 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919688d0-aa7c-469f-b0aa-71e31bc69798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:22.963328 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-919688d0-aa7c-469f-b0aa-71e31bc69798 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:49:22.965432 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-919688d0-aa7c-469f-b0aa-71e31bc69798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:23.080683 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1258/2523] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:49:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:49:23.121122 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c22e25-9c0a-47cb-b896-cd8522ce5476 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:23.122791 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-d6c22e25-9c0a-47cb-b896-cd8522ce5476 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:23.252912 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c22e25-9c0a-47cb-b896-cd8522ce5476 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:23.257025 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d879dc33-87b9-4b75-af0a-d7ea271db82e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:23.257851 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d879dc33-87b9-4b75-af0a-d7ea271db82e service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:23.258202 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d879dc33-87b9-4b75-af0a-d7ea271db82e service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:23.258607 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d879dc33-87b9-4b75-af0a-d7ea271db82e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:23.283944 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d879dc33-87b9-4b75-af0a-d7ea271db82e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:23.287950 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d879dc33-87b9-4b75-af0a-d7ea271db82e service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:23.301236 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1266/2524] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:49:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:49:23.770212 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919688d0-aa7c-469f-b0aa-71e31bc69798 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:23.772661 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-919688d0-aa7c-469f-b0aa-71e31bc69798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:23.799049 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919688d0-aa7c-469f-b0aa-71e31bc69798 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:23.802989 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaa4b92-489c-4078-9ecc-82680270ad2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:23.803908 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaa4b92-489c-4078-9ecc-82680270ad2a service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:23.804371 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaa4b92-489c-4078-9ecc-82680270ad2a service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:23.805328 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaa4b92-489c-4078-9ecc-82680270ad2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:23.815590 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-3eaa4b92-489c-4078-9ecc-82680270ad2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:23.844178 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaa4b92-489c-4078-9ecc-82680270ad2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:23.846931 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaa4b92-489c-4078-9ecc-82680270ad2a service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:23.872290 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1259/2525] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:49:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:49:24.506907 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d879dc33-87b9-4b75-af0a-d7ea271db82e service neutron] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:24.508983 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-d879dc33-87b9-4b75-af0a-d7ea271db82e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:24.530200 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d879dc33-87b9-4b75-af0a-d7ea271db82e service neutron] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:24.542933 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d400a25b-d460-4ebf-86c2-36d4a020c96b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:24.543687 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d400a25b-d460-4ebf-86c2-36d4a020c96b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:24.544485 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d400a25b-d460-4ebf-86c2-36d4a020c96b admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:24.544848 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d400a25b-d460-4ebf-86c2-36d4a020c96b admin admin] PATH_INFO: `/v3/projects/c15aca9ee17541a58651a57d2472af0b` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:24.551187 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d400a25b-d460-4ebf-86c2-36d4a020c96b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c15aca9ee17541a58651a57d2472af0b)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:24.554769 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d400a25b-d460-4ebf-86c2-36d4a020c96b admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:24.682071 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-d400a25b-d460-4ebf-86c2-36d4a020c96b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c15aca9ee17541a58651a57d2472af0b', 'request_id': 'req-d400a25b-d460-4ebf-86c2-36d4a020c96b'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:24.697966 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-d400a25b-d460-4ebf-86c2-36d4a020c96b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c15aca9ee17541a58651a57d2472af0b', 'request_id': 'req-d400a25b-d460-4ebf-86c2-36d4a020c96b'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:24.722645 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-d400a25b-d460-4ebf-86c2-36d4a020c96b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c15aca9ee17541a58651a57d2472af0b', 'request_id': 'req-d400a25b-d460-4ebf-86c2-36d4a020c96b'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:24.751107 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-d400a25b-d460-4ebf-86c2-36d4a020c96b admin admin] The token cache is being invalidate because project c15aca9ee17541a58651a57d2472af0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81617) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:49:24.751796 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-d400a25b-d460-4ebf-86c2-36d4a020c96b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:24.753185 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-d400a25b-d460-4ebf-86c2-36d4a020c96b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:24.770898 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1267/2526] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:49:24 2022] DELETE /identity/v3/projects/c15aca9ee17541a58651a57d2472af0b => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:49:24.772701 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaa4b92-489c-4078-9ecc-82680270ad2a service neutron] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:24.798209 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaa4b92-489c-4078-9ecc-82680270ad2a service neutron] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:24.934049 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aa737e-33f0-4627-bb1d-71913b9d1176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:24.934049 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa737e-33f0-4627-bb1d-71913b9d1176 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:24.934359 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa737e-33f0-4627-bb1d-71913b9d1176 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:24.934646 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa737e-33f0-4627-bb1d-71913b9d1176 admin admin] PATH_INFO: `/v3/users/9d5167905fe043b8b5c58cd783b9badc` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:24.935836 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aa737e-33f0-4627-bb1d-71913b9d1176 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d5167905fe043b8b5c58cd783b9badc)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:24.939818 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aa737e-33f0-4627-bb1d-71913b9d1176 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:25.020398 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:25.066135 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-f2aa737e-33f0-4627-bb1d-71913b9d1176 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d5167905fe043b8b5c58cd783b9badc'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:25.079841 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-f2aa737e-33f0-4627-bb1d-71913b9d1176 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d5167905fe043b8b5c58cd783b9badc'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:25.105070 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-f2aa737e-33f0-4627-bb1d-71913b9d1176 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d5167905fe043b8b5c58cd783b9badc'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:25.126912 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-f2aa737e-33f0-4627-bb1d-71913b9d1176 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d5167905fe043b8b5c58cd783b9badc'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:25.127558 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-f2aa737e-33f0-4627-bb1d-71913b9d1176 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d5167905fe043b8b5c58cd783b9badc'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:25.139705 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1260/2527] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:49:24 2022] DELETE /identity/v3/users/9d5167905fe043b8b5c58cd783b9badc => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:49:25.885177 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d400a25b-d460-4ebf-86c2-36d4a020c96b admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:25.886685 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-d400a25b-d460-4ebf-86c2-36d4a020c96b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:25.913993 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d400a25b-d460-4ebf-86c2-36d4a020c96b admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:25.963261 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:26.055188 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eb28d3-3ad3-4fdf-9d79-5165f9db9d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:26.055891 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eb28d3-3ad3-4fdf-9d79-5165f9db9d6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:26.056149 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eb28d3-3ad3-4fdf-9d79-5165f9db9d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:26.056728 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eb28d3-3ad3-4fdf-9d79-5165f9db9d6d admin admin] PATH_INFO: `/v3/projects/79a46b99b1b04cf29fc19a2d8aa827cd` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:26.062746 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eb28d3-3ad3-4fdf-9d79-5165f9db9d6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=79a46b99b1b04cf29fc19a2d8aa827cd)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:26.071116 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eb28d3-3ad3-4fdf-9d79-5165f9db9d6d admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:26.233060 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-29eb28d3-3ad3-4fdf-9d79-5165f9db9d6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79a46b99b1b04cf29fc19a2d8aa827cd', 'request_id': 'req-29eb28d3-3ad3-4fdf-9d79-5165f9db9d6d'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:26.249992 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-29eb28d3-3ad3-4fdf-9d79-5165f9db9d6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79a46b99b1b04cf29fc19a2d8aa827cd', 'request_id': 'req-29eb28d3-3ad3-4fdf-9d79-5165f9db9d6d'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:26.262597 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-29eb28d3-3ad3-4fdf-9d79-5165f9db9d6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79a46b99b1b04cf29fc19a2d8aa827cd', 'request_id': 'req-29eb28d3-3ad3-4fdf-9d79-5165f9db9d6d'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:26.293296 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-29eb28d3-3ad3-4fdf-9d79-5165f9db9d6d admin admin] The token cache is being invalidate because project 79a46b99b1b04cf29fc19a2d8aa827cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81617) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:49:26.293496 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-29eb28d3-3ad3-4fdf-9d79-5165f9db9d6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:26.296260 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-29eb28d3-3ad3-4fdf-9d79-5165f9db9d6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:26.299573 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1268/2528] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:49:25 2022] DELETE /identity/v3/projects/79a46b99b1b04cf29fc19a2d8aa827cd => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:49:26.304172 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3442e24-6f81-4041-918e-65018444ccae None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:26.304769 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3442e24-6f81-4041-918e-65018444ccae None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:26.304976 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3442e24-6f81-4041-918e-65018444ccae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:26.342168 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-a3442e24-6f81-4041-918e-65018444ccae None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:49:26.343793 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-a3442e24-6f81-4041-918e-65018444ccae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:26.505896 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1261/2529] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:49:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:49:26.515503 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d402e04-1893-4295-a34c-95748c2169d4 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:26.515879 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d402e04-1893-4295-a34c-95748c2169d4 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:26.516113 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d402e04-1893-4295-a34c-95748c2169d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:26.586316 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-1d402e04-1893-4295-a34c-95748c2169d4 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:49:26.589199 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-1d402e04-1893-4295-a34c-95748c2169d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:26.612108 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1269/2530] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:49:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:49:26.616458 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3442e24-6f81-4041-918e-65018444ccae None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:26.629731 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3442e24-6f81-4041-918e-65018444ccae None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:26.641663 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84803f93-2da8-4fa1-9f89-cdcd4251d5e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:26.642291 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84803f93-2da8-4fa1-9f89-cdcd4251d5e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:26.642593 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84803f93-2da8-4fa1-9f89-cdcd4251d5e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:26.642864 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84803f93-2da8-4fa1-9f89-cdcd4251d5e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:26.643518 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84803f93-2da8-4fa1-9f89-cdcd4251d5e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:49:26.643818 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84803f93-2da8-4fa1-9f89-cdcd4251d5e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:26.645402 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84803f93-2da8-4fa1-9f89-cdcd4251d5e4 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:26.680493 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1262/2531] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:49:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:49:26.685189 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d402e04-1893-4295-a34c-95748c2169d4 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:26.694506 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d402e04-1893-4295-a34c-95748c2169d4 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:26.712582 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a076fdfe-d1fb-4c48-b26d-eb99ef3f9d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:26.714549 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a076fdfe-d1fb-4c48-b26d-eb99ef3f9d02 admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:26.716712 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a076fdfe-d1fb-4c48-b26d-eb99ef3f9d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:26.716993 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a076fdfe-d1fb-4c48-b26d-eb99ef3f9d02 admin admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:26.717749 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a076fdfe-d1fb-4c48-b26d-eb99ef3f9d02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:26.720040 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a076fdfe-d1fb-4c48-b26d-eb99ef3f9d02 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:26.789630 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84803f93-2da8-4fa1-9f89-cdcd4251d5e4 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:26.800073 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1270/2532] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:49:26 2022] POST /identity/v3/projects => generated 375 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:49:26.854606 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84803f93-2da8-4fa1-9f89-cdcd4251d5e4 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:27.052707 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91303006-edfd-4eed-939b-36927904ab44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:27.058691 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91303006-edfd-4eed-939b-36927904ab44 admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:27.058691 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91303006-edfd-4eed-939b-36927904ab44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:27.061393 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91303006-edfd-4eed-939b-36927904ab44 admin admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:27.062941 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91303006-edfd-4eed-939b-36927904ab44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:27.069481 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91303006-edfd-4eed-939b-36927904ab44 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:27.105064 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1263/2533] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:49:26 2022] POST /identity/v3/users => generated 288 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 16:49:27.108898 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a076fdfe-d1fb-4c48-b26d-eb99ef3f9d02 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:27.132049 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a076fdfe-d1fb-4c48-b26d-eb99ef3f9d02 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:27.138338 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f281cf45-8c24-4de9-9195-b29f46bbfb22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:27.139641 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f281cf45-8c24-4de9-9195-b29f46bbfb22 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:27.140293 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f281cf45-8c24-4de9-9195-b29f46bbfb22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:27.140745 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f281cf45-8c24-4de9-9195-b29f46bbfb22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:27.141778 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f281cf45-8c24-4de9-9195-b29f46bbfb22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:27.144555 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f281cf45-8c24-4de9-9195-b29f46bbfb22 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:27.168538 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1271/2534] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:49:27 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:49:27.170094 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91303006-edfd-4eed-939b-36927904ab44 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:27.198279 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91303006-edfd-4eed-939b-36927904ab44 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:27.203347 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ab2e4d-985d-4189-802f-397bfc107063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:27.204749 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ab2e4d-985d-4189-802f-397bfc107063 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:27.204749 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ab2e4d-985d-4189-802f-397bfc107063 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:27.204749 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ab2e4d-985d-4189-802f-397bfc107063 admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:27.205275 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ab2e4d-985d-4189-802f-397bfc107063 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:27.207116 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ab2e4d-985d-4189-802f-397bfc107063 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:27.224191 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1264/2535] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:49:27 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:49:27.225822 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f281cf45-8c24-4de9-9195-b29f46bbfb22 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:27.238593 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f281cf45-8c24-4de9-9195-b29f46bbfb22 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:27.248900 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbeb2cb-e56a-498a-ae1f-13a4d8eea84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:27.249692 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbeb2cb-e56a-498a-ae1f-13a4d8eea84b admin admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:27.250031 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbeb2cb-e56a-498a-ae1f-13a4d8eea84b admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:27.250422 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbeb2cb-e56a-498a-ae1f-13a4d8eea84b admin admin] PATH_INFO: `/v3/projects/aaf67aec835845268f60740016d89ba0/users/2d3b60e900c14e4596c9ee837a6ddbea/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:27.296446 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbeb2cb-e56a-498a-ae1f-13a4d8eea84b admin admin] RBAC: Authorizing `identity:create_grant(project_id=aaf67aec835845268f60740016d89ba0, user_id=2d3b60e900c14e4596c9ee837a6ddbea, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:27.299517 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbeb2cb-e56a-498a-ae1f-13a4d8eea84b admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:27.338105 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1272/2536] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:49:27 2022] PUT /identity/v3/projects/aaf67aec835845268f60740016d89ba0/users/2d3b60e900c14e4596c9ee837a6ddbea/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:49:27.344001 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57338519-28b4-4331-85a1-f0e4d66e0e24 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:27.345129 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57338519-28b4-4331-85a1-f0e4d66e0e24 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:27.346107 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57338519-28b4-4331-85a1-f0e4d66e0e24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:27.392006 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:27.407028 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-57338519-28b4-4331-85a1-f0e4d66e0e24 None None] MFA Rules not processed for user `2d3b60e900c14e4596c9ee837a6ddbea`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:49:27.413059 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-57338519-28b4-4331-85a1-f0e4d66e0e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:27.568485 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1265/2537] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:49:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:49:27.577516 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbeb2cb-e56a-498a-ae1f-13a4d8eea84b admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:27.579053 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-4cbeb2cb-e56a-498a-ae1f-13a4d8eea84b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:27.729660 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbeb2cb-e56a-498a-ae1f-13a4d8eea84b admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:27.736036 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54c1264-f743-4ef1-84b5-5c7fc658bee9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:27.736682 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54c1264-f743-4ef1-84b5-5c7fc658bee9 service cinder] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:27.737668 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54c1264-f743-4ef1-84b5-5c7fc658bee9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:27.737970 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54c1264-f743-4ef1-84b5-5c7fc658bee9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:27.752646 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54c1264-f743-4ef1-84b5-5c7fc658bee9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:27.754125 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54c1264-f743-4ef1-84b5-5c7fc658bee9 service cinder] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:27.781352 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1273/2538] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:49:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:49:28.817198 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57338519-28b4-4331-85a1-f0e4d66e0e24 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:28.818990 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-57338519-28b4-4331-85a1-f0e4d66e0e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:29.093696 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57338519-28b4-4331-85a1-f0e4d66e0e24 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:29.121652 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93782d50-cd63-4e75-85a5-d6e8bbef87f0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1a985495ed48a4acc48e2e6fc7f977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:29.123023 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93782d50-cd63-4e75-85a5-d6e8bbef87f0 service placement] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:29.124008 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93782d50-cd63-4e75-85a5-d6e8bbef87f0 service placement] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:29.124807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93782d50-cd63-4e75-85a5-d6e8bbef87f0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:29.135119 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-93782d50-cd63-4e75-85a5-d6e8bbef87f0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:29.163325 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93782d50-cd63-4e75-85a5-d6e8bbef87f0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:29.166810 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93782d50-cd63-4e75-85a5-d6e8bbef87f0 service placement] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:29.186139 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1266/2539] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:49:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:49:30.797989 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54c1264-f743-4ef1-84b5-5c7fc658bee9 service cinder] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:30.799514 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-f54c1264-f743-4ef1-84b5-5c7fc658bee9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:30.816686 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:31.028142 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54c1264-f743-4ef1-84b5-5c7fc658bee9 service cinder] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:31.053090 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a615f4cd-2ce5-4196-ad2f-02b165edbf66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:31.053766 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a615f4cd-2ce5-4196-ad2f-02b165edbf66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:31.054049 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a615f4cd-2ce5-4196-ad2f-02b165edbf66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:31.054472 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a615f4cd-2ce5-4196-ad2f-02b165edbf66 admin admin] PATH_INFO: `/v3/users/8cd9c727c0b444cb83497ea4f78e24d0` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:31.055841 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a615f4cd-2ce5-4196-ad2f-02b165edbf66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cd9c727c0b444cb83497ea4f78e24d0)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:31.058104 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a615f4cd-2ce5-4196-ad2f-02b165edbf66 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:31.193048 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-a615f4cd-2ce5-4196-ad2f-02b165edbf66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cd9c727c0b444cb83497ea4f78e24d0'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:31.195722 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-a615f4cd-2ce5-4196-ad2f-02b165edbf66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cd9c727c0b444cb83497ea4f78e24d0'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:31.209511 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-a615f4cd-2ce5-4196-ad2f-02b165edbf66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cd9c727c0b444cb83497ea4f78e24d0'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:31.238346 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-a615f4cd-2ce5-4196-ad2f-02b165edbf66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cd9c727c0b444cb83497ea4f78e24d0'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:31.239544 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-a615f4cd-2ce5-4196-ad2f-02b165edbf66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cd9c727c0b444cb83497ea4f78e24d0'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:31.280281 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1274/2540] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:49:30 2022] DELETE /identity/v3/users/8cd9c727c0b444cb83497ea4f78e24d0 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:49:32.082684 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93782d50-cd63-4e75-85a5-d6e8bbef87f0 service placement] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:32.103104 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93782d50-cd63-4e75-85a5-d6e8bbef87f0 service placement] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:32.164210 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a615f4cd-2ce5-4196-ad2f-02b165edbf66 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:32.166624 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-a615f4cd-2ce5-4196-ad2f-02b165edbf66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:32.285592 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77944a4-48f7-44ad-9004-f07447821353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:32.286696 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77944a4-48f7-44ad-9004-f07447821353 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:32.288226 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77944a4-48f7-44ad-9004-f07447821353 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:32.288764 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77944a4-48f7-44ad-9004-f07447821353 admin admin] PATH_INFO: `/v3/users/2d3b60e900c14e4596c9ee837a6ddbea` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:32.291075 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77944a4-48f7-44ad-9004-f07447821353 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d3b60e900c14e4596c9ee837a6ddbea)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:32.296877 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77944a4-48f7-44ad-9004-f07447821353 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:32.353630 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a615f4cd-2ce5-4196-ad2f-02b165edbf66 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:32.360825 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cd3916-cb34-45b5-bd7d-cebaa2307fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:32.361599 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cd3916-cb34-45b5-bd7d-cebaa2307fd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:32.361995 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cd3916-cb34-45b5-bd7d-cebaa2307fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:32.364882 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cd3916-cb34-45b5-bd7d-cebaa2307fd9 admin admin] PATH_INFO: `/v3/projects/0e35510b626e46aaae189f31b27ed663` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:32.367739 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cd3916-cb34-45b5-bd7d-cebaa2307fd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e35510b626e46aaae189f31b27ed663)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:32.371335 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cd3916-cb34-45b5-bd7d-cebaa2307fd9 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:32.379294 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:32.442361 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-b77944a4-48f7-44ad-9004-f07447821353 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d3b60e900c14e4596c9ee837a6ddbea'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:32.479570 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:32.482627 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-b77944a4-48f7-44ad-9004-f07447821353 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d3b60e900c14e4596c9ee837a6ddbea'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:32.547353 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-b77944a4-48f7-44ad-9004-f07447821353 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d3b60e900c14e4596c9ee837a6ddbea'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:32.557196 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-43cd3916-cb34-45b5-bd7d-cebaa2307fd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e35510b626e46aaae189f31b27ed663', 'request_id': 'req-43cd3916-cb34-45b5-bd7d-cebaa2307fd9'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:32.569478 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-b77944a4-48f7-44ad-9004-f07447821353 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d3b60e900c14e4596c9ee837a6ddbea'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:32.574160 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-b77944a4-48f7-44ad-9004-f07447821353 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d3b60e900c14e4596c9ee837a6ddbea'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:32.579870 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-43cd3916-cb34-45b5-bd7d-cebaa2307fd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e35510b626e46aaae189f31b27ed663', 'request_id': 'req-43cd3916-cb34-45b5-bd7d-cebaa2307fd9'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:32.599740 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1267/2541] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:49:32 2022] DELETE /identity/v3/users/2d3b60e900c14e4596c9ee837a6ddbea => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:49:32.602206 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf36978-1478-4a0d-a508-5c0eb023a3ec None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:32.604993 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf36978-1478-4a0d-a508-5c0eb023a3ec None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:32.605954 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf36978-1478-4a0d-a508-5c0eb023a3ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:32.617514 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-43cd3916-cb34-45b5-bd7d-cebaa2307fd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e35510b626e46aaae189f31b27ed663', 'request_id': 'req-43cd3916-cb34-45b5-bd7d-cebaa2307fd9'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:32.656506 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-43cd3916-cb34-45b5-bd7d-cebaa2307fd9 admin admin] The token cache is being invalidate because project 0e35510b626e46aaae189f31b27ed663 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81617) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:49:32.657141 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-43cd3916-cb34-45b5-bd7d-cebaa2307fd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:32.660118 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-43cd3916-cb34-45b5-bd7d-cebaa2307fd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:32.670173 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1275/2542] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:49:32 2022] DELETE /identity/v3/projects/0e35510b626e46aaae189f31b27ed663 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:49:32.676060 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6861d14b-f26c-4689-922a-62bf036b2e83 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:32.676637 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6861d14b-f26c-4689-922a-62bf036b2e83 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:32.679356 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6861d14b-f26c-4689-922a-62bf036b2e83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:32.684870 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-bdf36978-1478-4a0d-a508-5c0eb023a3ec None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:49:32.686493 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-bdf36978-1478-4a0d-a508-5c0eb023a3ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:32.731449 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-6861d14b-f26c-4689-922a-62bf036b2e83 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:49:32.732988 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-6861d14b-f26c-4689-922a-62bf036b2e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:32.816257 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1276/2543] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:49:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:49:32.828913 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8351ec-979d-4a18-b02a-114d23d37124 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:32.831338 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8351ec-979d-4a18-b02a-114d23d37124 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:32.831765 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8351ec-979d-4a18-b02a-114d23d37124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:32.864781 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1268/2544] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:49:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:49:32.878678 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-cf8351ec-979d-4a18-b02a-114d23d37124 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:49:32.879973 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-cf8351ec-979d-4a18-b02a-114d23d37124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:32.881484 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf36978-1478-4a0d-a508-5c0eb023a3ec None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:32.883824 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-bdf36978-1478-4a0d-a508-5c0eb023a3ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:32.902523 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1277/2545] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:49:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:49:32.902853 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8351ec-979d-4a18-b02a-114d23d37124 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:32.916602 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8351ec-979d-4a18-b02a-114d23d37124 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:32.922495 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73443c3f-5773-4f0a-afd3-66ba231b7aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:32.923130 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73443c3f-5773-4f0a-afd3-66ba231b7aa2 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:32.923421 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73443c3f-5773-4f0a-afd3-66ba231b7aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:32.923772 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73443c3f-5773-4f0a-afd3-66ba231b7aa2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:32.924450 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73443c3f-5773-4f0a-afd3-66ba231b7aa2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:49:32.924797 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73443c3f-5773-4f0a-afd3-66ba231b7aa2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:32.927634 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73443c3f-5773-4f0a-afd3-66ba231b7aa2 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:32.957896 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1278/2546] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:49:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:49:32.962983 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73443c3f-5773-4f0a-afd3-66ba231b7aa2 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:32.987676 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73443c3f-5773-4f0a-afd3-66ba231b7aa2 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:33.002482 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c5d317-27c9-4039-9aed-ee85dc6654a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:33.003409 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c5d317-27c9-4039-9aed-ee85dc6654a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:33.003905 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c5d317-27c9-4039-9aed-ee85dc6654a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:33.004568 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c5d317-27c9-4039-9aed-ee85dc6654a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:33.005584 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c5d317-27c9-4039-9aed-ee85dc6654a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:33.007913 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c5d317-27c9-4039-9aed-ee85dc6654a1 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:33.030983 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf36978-1478-4a0d-a508-5c0eb023a3ec None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:33.038325 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e2581c-ebc4-4160-a043-2a508b792660 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:33.039511 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e2581c-ebc4-4160-a043-2a508b792660 service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:33.039790 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e2581c-ebc4-4160-a043-2a508b792660 service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:33.040119 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e2581c-ebc4-4160-a043-2a508b792660 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:33.079077 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e2581c-ebc4-4160-a043-2a508b792660 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:33.081125 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e2581c-ebc4-4160-a043-2a508b792660 service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:33.108438 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1279/2547] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:49:32 2022] POST /identity/v3/projects => generated 367 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 16:49:33.110292 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c5d317-27c9-4039-9aed-ee85dc6654a1 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:33.111452 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1269/2548] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:49:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:49:33.231012 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c5d317-27c9-4039-9aed-ee85dc6654a1 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:33.236975 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b313f46-0d99-4e89-9ae9-5e221808246a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:33.237606 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b313f46-0d99-4e89-9ae9-5e221808246a admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:33.237940 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b313f46-0d99-4e89-9ae9-5e221808246a admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:33.238163 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b313f46-0d99-4e89-9ae9-5e221808246a admin admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:33.238864 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b313f46-0d99-4e89-9ae9-5e221808246a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:33.242413 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b313f46-0d99-4e89-9ae9-5e221808246a admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:33.266095 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=89,keystone:DELETE=6,keystone:INSERT=5 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:33.278089 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e2581c-ebc4-4160-a043-2a508b792660 service neutron] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:33.286036 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1280/2549] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:49:33 2022] POST /identity/v3/users => generated 284 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 16:49:33.289870 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e2581c-ebc4-4160-a043-2a508b792660 service neutron] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:33.298774 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8bddff-a415-4728-82ae-f5287b2ace41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:33.299395 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8bddff-a415-4728-82ae-f5287b2ace41 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:33.299676 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8bddff-a415-4728-82ae-f5287b2ace41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:33.299951 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8bddff-a415-4728-82ae-f5287b2ace41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:33.300662 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8bddff-a415-4728-82ae-f5287b2ace41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:33.302317 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8bddff-a415-4728-82ae-f5287b2ace41 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:33.321414 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1270/2550] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:49:33 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:49:33.322805 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b313f46-0d99-4e89-9ae9-5e221808246a admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:33.336194 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b313f46-0d99-4e89-9ae9-5e221808246a admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:33.343677 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf821e2-e8a4-410c-b78a-aa530b88829e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:33.344911 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf821e2-e8a4-410c-b78a-aa530b88829e admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:33.345332 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf821e2-e8a4-410c-b78a-aa530b88829e admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:33.345672 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf821e2-e8a4-410c-b78a-aa530b88829e admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:33.346485 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf821e2-e8a4-410c-b78a-aa530b88829e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:33.348380 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf821e2-e8a4-410c-b78a-aa530b88829e admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:33.363962 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1281/2551] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:49:33 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:49:33.365630 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8bddff-a415-4728-82ae-f5287b2ace41 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:33.381975 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8bddff-a415-4728-82ae-f5287b2ace41 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:33.387929 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97982406-df1a-4504-8143-ec335b08b582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:33.388661 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97982406-df1a-4504-8143-ec335b08b582 admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:33.389466 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97982406-df1a-4504-8143-ec335b08b582 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:33.389779 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97982406-df1a-4504-8143-ec335b08b582 admin admin] PATH_INFO: `/v3/projects/f88505764705431cbea1bb555f198999/users/772919ee28d54a4b84b8237d04a65879/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:33.410684 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97982406-df1a-4504-8143-ec335b08b582 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f88505764705431cbea1bb555f198999, user_id=772919ee28d54a4b84b8237d04a65879, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:33.413546 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97982406-df1a-4504-8143-ec335b08b582 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:33.455610 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d3e67-f896-46a2-9678-90b1706bc186 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:33.455879 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1271/2552] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:49:33 2022] PUT /identity/v3/projects/f88505764705431cbea1bb555f198999/users/772919ee28d54a4b84b8237d04a65879/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:49:33.456470 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d3e67-f896-46a2-9678-90b1706bc186 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:33.456753 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d3e67-f896-46a2-9678-90b1706bc186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:33.518762 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-503d3e67-f896-46a2-9678-90b1706bc186 None None] MFA Rules not processed for user `772919ee28d54a4b84b8237d04a65879`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:49:33.520096 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-503d3e67-f896-46a2-9678-90b1706bc186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:33.652182 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1282/2553] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:49:33 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:49:33.654130 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97982406-df1a-4504-8143-ec335b08b582 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:33.680467 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:33.760982 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97982406-df1a-4504-8143-ec335b08b582 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:33.777226 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab5f0e3-3c2e-45f1-bf36-fbf0454b2491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:33.778431 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab5f0e3-3c2e-45f1-bf36-fbf0454b2491 admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:33.779115 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab5f0e3-3c2e-45f1-bf36-fbf0454b2491 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:33.779712 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab5f0e3-3c2e-45f1-bf36-fbf0454b2491 admin admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:33.781244 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab5f0e3-3c2e-45f1-bf36-fbf0454b2491 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:33.783891 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab5f0e3-3c2e-45f1-bf36-fbf0454b2491 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:33.863298 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d3e67-f896-46a2-9678-90b1706bc186 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:33.886235 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1272/2554] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:49:33 2022] POST /identity/v3/projects => generated 369 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:49:33.894685 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d3e67-f896-46a2-9678-90b1706bc186 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:34.045005 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c925b1-5f5e-46d3-9ed0-c7cc558ece1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:34.045636 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c925b1-5f5e-46d3-9ed0-c7cc558ece1c admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:34.045925 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c925b1-5f5e-46d3-9ed0-c7cc558ece1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:34.046195 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c925b1-5f5e-46d3-9ed0-c7cc558ece1c admin admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:34.046981 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c925b1-5f5e-46d3-9ed0-c7cc558ece1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:34.062278 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c925b1-5f5e-46d3-9ed0-c7cc558ece1c admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:34.170829 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab5f0e3-3c2e-45f1-bf36-fbf0454b2491 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:34.178901 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1283/2555] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:49:33 2022] POST /identity/v3/users => generated 291 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:49:34.192345 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c925b1-5f5e-46d3-9ed0-c7cc558ece1c admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:34.200688 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-82c925b1-5f5e-46d3-9ed0-c7cc558ece1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:34.204985 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab5f0e3-3c2e-45f1-bf36-fbf0454b2491 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:34.214669 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f077cf1-948f-404c-ac18-237d3cd683bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:34.216542 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f077cf1-948f-404c-ac18-237d3cd683bb admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:34.216762 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f077cf1-948f-404c-ac18-237d3cd683bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:34.217048 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f077cf1-948f-404c-ac18-237d3cd683bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:34.218128 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f077cf1-948f-404c-ac18-237d3cd683bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:34.219423 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f077cf1-948f-404c-ac18-237d3cd683bb admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:34.235431 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1273/2556] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:49:34 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:49:34.243631 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f077cf1-948f-404c-ac18-237d3cd683bb admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:34.254529 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c925b1-5f5e-46d3-9ed0-c7cc558ece1c admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:34.274502 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f077cf1-948f-404c-ac18-237d3cd683bb admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:34.277347 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf1dea5-9f32-4eef-b455-7fc8be5300a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:34.278798 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf1dea5-9f32-4eef-b455-7fc8be5300a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:34.279266 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf1dea5-9f32-4eef-b455-7fc8be5300a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:34.279709 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf1dea5-9f32-4eef-b455-7fc8be5300a6 admin admin] PATH_INFO: `/v3/projects/aaf67aec835845268f60740016d89ba0` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:34.283126 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43488b6a-3f9c-4580-9d37-d610aa8a2d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:34.283852 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43488b6a-3f9c-4580-9d37-d610aa8a2d04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:34.284141 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43488b6a-3f9c-4580-9d37-d610aa8a2d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:34.284566 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43488b6a-3f9c-4580-9d37-d610aa8a2d04 admin admin] PATH_INFO: `/v3/projects/b8e7160e42444ea8ace7e7a0fe159e55/users/837c0a783a7e4315a71dad78757ae92d/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:34.285810 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf1dea5-9f32-4eef-b455-7fc8be5300a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aaf67aec835845268f60740016d89ba0)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:34.289104 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf1dea5-9f32-4eef-b455-7fc8be5300a6 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:34.323646 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43488b6a-3f9c-4580-9d37-d610aa8a2d04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8e7160e42444ea8ace7e7a0fe159e55, user_id=837c0a783a7e4315a71dad78757ae92d, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:34.327452 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43488b6a-3f9c-4580-9d37-d610aa8a2d04 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:34.357320 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1274/2557] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:49:34 2022] PUT /identity/v3/projects/b8e7160e42444ea8ace7e7a0fe159e55/users/837c0a783a7e4315a71dad78757ae92d/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:49:34.358715 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43488b6a-3f9c-4580-9d37-d610aa8a2d04 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:34.408627 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-4bf1dea5-9f32-4eef-b455-7fc8be5300a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aaf67aec835845268f60740016d89ba0', 'request_id': 'req-4bf1dea5-9f32-4eef-b455-7fc8be5300a6'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:34.419344 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-4bf1dea5-9f32-4eef-b455-7fc8be5300a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aaf67aec835845268f60740016d89ba0', 'request_id': 'req-4bf1dea5-9f32-4eef-b455-7fc8be5300a6'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:34.438650 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-4bf1dea5-9f32-4eef-b455-7fc8be5300a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aaf67aec835845268f60740016d89ba0', 'request_id': 'req-4bf1dea5-9f32-4eef-b455-7fc8be5300a6'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:34.443839 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:34.462980 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-4bf1dea5-9f32-4eef-b455-7fc8be5300a6 admin admin] The token cache is being invalidate because project aaf67aec835845268f60740016d89ba0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81617) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:49:34.463621 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-4bf1dea5-9f32-4eef-b455-7fc8be5300a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:34.464998 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-4bf1dea5-9f32-4eef-b455-7fc8be5300a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:34.471335 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1284/2558] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:49:34 2022] DELETE /identity/v3/projects/aaf67aec835845268f60740016d89ba0 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:49:34.477061 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed68843c-9b36-4aee-8d59-59cdcc397393 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:34.477346 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed68843c-9b36-4aee-8d59-59cdcc397393 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:34.477625 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed68843c-9b36-4aee-8d59-59cdcc397393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:34.481696 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43488b6a-3f9c-4580-9d37-d610aa8a2d04 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:34.543283 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-ed68843c-9b36-4aee-8d59-59cdcc397393 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:49:34.544665 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-ed68843c-9b36-4aee-8d59-59cdcc397393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:34.618435 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cc44b7-0fed-4acf-b6cd-3cf1226cdf3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:34.619184 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cc44b7-0fed-4acf-b6cd-3cf1226cdf3b admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:34.619564 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cc44b7-0fed-4acf-b6cd-3cf1226cdf3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:34.622195 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cc44b7-0fed-4acf-b6cd-3cf1226cdf3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:34.622195 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cc44b7-0fed-4acf-b6cd-3cf1226cdf3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:34.623224 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cc44b7-0fed-4acf-b6cd-3cf1226cdf3b admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:34.631864 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1285/2559] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:49:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:49:34.638337 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1275/2560] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:49:34 2022] GET /identity/v3/roles => generated 1420 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:49:34.641336 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68fa78c-c5c0-4e3e-a07b-457553492576 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:34.641336 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68fa78c-c5c0-4e3e-a07b-457553492576 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:34.641336 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68fa78c-c5c0-4e3e-a07b-457553492576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:34.644986 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cc44b7-0fed-4acf-b6cd-3cf1226cdf3b admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:34.647543 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-81cc44b7-0fed-4acf-b6cd-3cf1226cdf3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:34.665922 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cc44b7-0fed-4acf-b6cd-3cf1226cdf3b admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:34.672386 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-a68fa78c-c5c0-4e3e-a07b-457553492576 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:49:34.673744 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-a68fa78c-c5c0-4e3e-a07b-457553492576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:34.686238 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf03e53-10c7-4b53-bd01-5d103e0aee29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:34.686534 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1286/2561] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:49:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:49:34.686653 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf03e53-10c7-4b53-bd01-5d103e0aee29 admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:34.686653 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf03e53-10c7-4b53-bd01-5d103e0aee29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:34.686653 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf03e53-10c7-4b53-bd01-5d103e0aee29 admin admin] PATH_INFO: `/v3/projects/b8e7160e42444ea8ace7e7a0fe159e55/users/837c0a783a7e4315a71dad78757ae92d/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:34.688224 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68fa78c-c5c0-4e3e-a07b-457553492576 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:34.688572 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf03e53-10c7-4b53-bd01-5d103e0aee29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8e7160e42444ea8ace7e7a0fe159e55, user_id=837c0a783a7e4315a71dad78757ae92d, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:34.691348 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf03e53-10c7-4b53-bd01-5d103e0aee29 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:34.703467 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68fa78c-c5c0-4e3e-a07b-457553492576 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:34.707000 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:34.724122 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1276/2562] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:49:34 2022] PUT /identity/v3/projects/b8e7160e42444ea8ace7e7a0fe159e55/users/837c0a783a7e4315a71dad78757ae92d/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:49:34.738045 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8504c62f-c12e-4e52-ba1d-186b02dde5c3 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:34.739286 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8504c62f-c12e-4e52-ba1d-186b02dde5c3 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:34.739698 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8504c62f-c12e-4e52-ba1d-186b02dde5c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:34.779655 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-8504c62f-c12e-4e52-ba1d-186b02dde5c3 None None] MFA Rules not processed for user `837c0a783a7e4315a71dad78757ae92d`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:49:34.783151 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-8504c62f-c12e-4e52-ba1d-186b02dde5c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:34.830900 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d8aa74-6533-4e50-bc0e-1cb5d4d41143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:34.831532 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8aa74-6533-4e50-bc0e-1cb5d4d41143 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:34.831815 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8aa74-6533-4e50-bc0e-1cb5d4d41143 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:34.832093 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8aa74-6533-4e50-bc0e-1cb5d4d41143 admin admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:34.832815 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d8aa74-6533-4e50-bc0e-1cb5d4d41143 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:49:34.833066 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d8aa74-6533-4e50-bc0e-1cb5d4d41143 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:34.834716 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d8aa74-6533-4e50-bc0e-1cb5d4d41143 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:34.861474 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1287/2563] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:49:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:49:34.862503 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d8aa74-6533-4e50-bc0e-1cb5d4d41143 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:34.885900 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d8aa74-6533-4e50-bc0e-1cb5d4d41143 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:34.905818 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030d2ab6-5c85-4bed-bb18-20babbd09eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:34.907520 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030d2ab6-5c85-4bed-bb18-20babbd09eea admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:34.907833 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030d2ab6-5c85-4bed-bb18-20babbd09eea admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:34.908112 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030d2ab6-5c85-4bed-bb18-20babbd09eea admin admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:34.912285 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030d2ab6-5c85-4bed-bb18-20babbd09eea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:34.916071 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030d2ab6-5c85-4bed-bb18-20babbd09eea admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:34.980079 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1288/2564] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:49:34 2022] POST /identity/v3/projects => generated 381 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:49:34.983261 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030d2ab6-5c85-4bed-bb18-20babbd09eea admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:35.029445 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1277/2565] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:49:34 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:49:35.041973 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8504c62f-c12e-4e52-ba1d-186b02dde5c3 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:35.043930 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-8504c62f-c12e-4e52-ba1d-186b02dde5c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:35.117929 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:INSERT=2,keystone:DELETE=1,keystone:SELECT=97 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:35.124856 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030d2ab6-5c85-4bed-bb18-20babbd09eea admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:35.132415 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142d3970-a463-4960-a670-cadc71e1da98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:35.133216 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142d3970-a463-4960-a670-cadc71e1da98 admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:35.133724 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142d3970-a463-4960-a670-cadc71e1da98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:35.134114 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142d3970-a463-4960-a670-cadc71e1da98 admin admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:35.135004 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142d3970-a463-4960-a670-cadc71e1da98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:35.138930 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142d3970-a463-4960-a670-cadc71e1da98 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:35.151480 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8504c62f-c12e-4e52-ba1d-186b02dde5c3 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:35.177005 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1289/2566] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:49:34 2022] POST /identity/v3/users => generated 291 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:49:35.180183 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142d3970-a463-4960-a670-cadc71e1da98 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:35.200971 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142d3970-a463-4960-a670-cadc71e1da98 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:35.217368 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4e3f9d-3e23-40b4-bd19-ad6f57f718c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:35.218665 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e3f9d-3e23-40b4-bd19-ad6f57f718c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:35.219046 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e3f9d-3e23-40b4-bd19-ad6f57f718c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:35.219415 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e3f9d-3e23-40b4-bd19-ad6f57f718c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:35.220474 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4e3f9d-3e23-40b4-bd19-ad6f57f718c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:35.222890 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4e3f9d-3e23-40b4-bd19-ad6f57f718c6 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:35.239323 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eb0d0d-e730-4962-9373-0edc8e14b95e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:35.239983 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb0d0d-e730-4962-9373-0edc8e14b95e service cinder] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:35.240284 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb0d0d-e730-4962-9373-0edc8e14b95e service cinder] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:35.240564 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb0d0d-e730-4962-9373-0edc8e14b95e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:35.242983 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1290/2567] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:49:35 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:49:35.247092 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4e3f9d-3e23-40b4-bd19-ad6f57f718c6 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:35.263089 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4e3f9d-3e23-40b4-bd19-ad6f57f718c6 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:35.268075 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b284e00-c150-4215-a4b8-51873f219ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:35.269539 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b284e00-c150-4215-a4b8-51873f219ef8 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:35.271116 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b284e00-c150-4215-a4b8-51873f219ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:35.271618 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b284e00-c150-4215-a4b8-51873f219ef8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:35.274622 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b284e00-c150-4215-a4b8-51873f219ef8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:35.276980 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b284e00-c150-4215-a4b8-51873f219ef8 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:35.291411 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1291/2568] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:49:35 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:49:35.299949 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b284e00-c150-4215-a4b8-51873f219ef8 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:35.330063 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b284e00-c150-4215-a4b8-51873f219ef8 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:35.348902 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780e1ddb-9796-4fe5-90a9-a558a51f37ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:35.349589 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e1ddb-9796-4fe5-90a9-a558a51f37ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:35.349871 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e1ddb-9796-4fe5-90a9-a558a51f37ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:35.350145 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e1ddb-9796-4fe5-90a9-a558a51f37ba admin admin] PATH_INFO: `/v3/projects/e2e1e5be5977467fb5f7946d8c0729b1/users/59d27a486fa54430aa97694b65f69e36/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:35.364390 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:35.385992 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eb0d0d-e730-4962-9373-0edc8e14b95e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:35.387965 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780e1ddb-9796-4fe5-90a9-a558a51f37ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2e1e5be5977467fb5f7946d8c0729b1, user_id=59d27a486fa54430aa97694b65f69e36, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:35.388426 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eb0d0d-e730-4962-9373-0edc8e14b95e service cinder] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:35.391484 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780e1ddb-9796-4fe5-90a9-a558a51f37ba admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:35.412007 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1278/2569] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:49:35 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 16:49:35.427893 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1292/2570] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:49:35 2022] PUT /identity/v3/projects/e2e1e5be5977467fb5f7946d8c0729b1/users/59d27a486fa54430aa97694b65f69e36/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:49:35.429807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4162f15-403b-4745-8f81-3a0881296b70 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:35.430134 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4162f15-403b-4745-8f81-3a0881296b70 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:35.430583 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4162f15-403b-4745-8f81-3a0881296b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:35.502911 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-b4162f15-403b-4745-8f81-3a0881296b70 None None] MFA Rules not processed for user `59d27a486fa54430aa97694b65f69e36`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:49:35.504478 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-b4162f15-403b-4745-8f81-3a0881296b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:35.641843 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780e1ddb-9796-4fe5-90a9-a558a51f37ba admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:35.644346 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-780e1ddb-9796-4fe5-90a9-a558a51f37ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:35.728504 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1279/2571] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:49:35 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:49:35.733482 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4162f15-403b-4745-8f81-3a0881296b70 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:35.824044 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780e1ddb-9796-4fe5-90a9-a558a51f37ba admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:35.841279 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8292773c-e72d-4fdf-b01d-5f3ac3f71cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:35.841861 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8292773c-e72d-4fdf-b01d-5f3ac3f71cae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:35.842138 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8292773c-e72d-4fdf-b01d-5f3ac3f71cae admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:35.844531 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8292773c-e72d-4fdf-b01d-5f3ac3f71cae admin admin] PATH_INFO: `/v3/users/960704dfe18c4d739e90d00288724983` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:35.847021 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8292773c-e72d-4fdf-b01d-5f3ac3f71cae admin admin] RBAC: Authorizing `identity:delete_user(user_id=960704dfe18c4d739e90d00288724983)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:35.849198 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8292773c-e72d-4fdf-b01d-5f3ac3f71cae admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:35.899283 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4162f15-403b-4745-8f81-3a0881296b70 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:35.916651 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6ec738-7a03-4c85-93ee-f2c9d7cd26e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:35.917420 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ec738-7a03-4c85-93ee-f2c9d7cd26e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:35.917698 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ec738-7a03-4c85-93ee-f2c9d7cd26e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:35.919131 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ec738-7a03-4c85-93ee-f2c9d7cd26e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:35.919932 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6ec738-7a03-4c85-93ee-f2c9d7cd26e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:35.922344 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6ec738-7a03-4c85-93ee-f2c9d7cd26e7 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:35.970810 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1280/2572] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:49:35 2022] POST /identity/v3/projects => generated 385 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:49:35.972069 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6ec738-7a03-4c85-93ee-f2c9d7cd26e7 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:35.979385 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-8292773c-e72d-4fdf-b01d-5f3ac3f71cae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '960704dfe18c4d739e90d00288724983'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:35.982502 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-8292773c-e72d-4fdf-b01d-5f3ac3f71cae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '960704dfe18c4d739e90d00288724983'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:36.002133 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-8292773c-e72d-4fdf-b01d-5f3ac3f71cae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '960704dfe18c4d739e90d00288724983'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:36.023800 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-8292773c-e72d-4fdf-b01d-5f3ac3f71cae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '960704dfe18c4d739e90d00288724983'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:36.027006 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-8292773c-e72d-4fdf-b01d-5f3ac3f71cae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '960704dfe18c4d739e90d00288724983'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:36.054999 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1293/2573] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:49:35 2022] DELETE /identity/v3/users/960704dfe18c4d739e90d00288724983 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 16:49:36.094352 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6ec738-7a03-4c85-93ee-f2c9d7cd26e7 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:36.112572 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:36.221458 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c635353b-5c8c-4de4-ab67-4184a169e94d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:36.221869 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c635353b-5c8c-4de4-ab67-4184a169e94d admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:36.222151 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c635353b-5c8c-4de4-ab67-4184a169e94d admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:36.222533 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c635353b-5c8c-4de4-ab67-4184a169e94d admin admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:36.223202 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c635353b-5c8c-4de4-ab67-4184a169e94d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:36.225166 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c635353b-5c8c-4de4-ab67-4184a169e94d admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:36.253442 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1281/2574] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:49:35 2022] POST /identity/v3/users => generated 293 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:49:36.258883 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8292773c-e72d-4fdf-b01d-5f3ac3f71cae admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:36.261483 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-8292773c-e72d-4fdf-b01d-5f3ac3f71cae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:36.291985 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8292773c-e72d-4fdf-b01d-5f3ac3f71cae admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:36.303331 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc7d7a-c803-4f4b-800d-9966401223d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:36.304252 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc7d7a-c803-4f4b-800d-9966401223d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:36.305399 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc7d7a-c803-4f4b-800d-9966401223d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:36.305742 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc7d7a-c803-4f4b-800d-9966401223d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:36.306826 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffc7d7a-c803-4f4b-800d-9966401223d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:36.308573 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffc7d7a-c803-4f4b-800d-9966401223d3 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:36.331699 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1294/2575] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:49:36 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:49:36.336897 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c635353b-5c8c-4de4-ab67-4184a169e94d admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:36.357232 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc7d7a-c803-4f4b-800d-9966401223d3 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:36.361625 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-dffc7d7a-c803-4f4b-800d-9966401223d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:36.369195 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c635353b-5c8c-4de4-ab67-4184a169e94d admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:36.390934 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc7d7a-c803-4f4b-800d-9966401223d3 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:36.399039 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5344bca8-afd2-4f57-8e1f-8f349074983c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:36.401199 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5344bca8-afd2-4f57-8e1f-8f349074983c admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:36.402408 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5344bca8-afd2-4f57-8e1f-8f349074983c admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:36.402408 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5344bca8-afd2-4f57-8e1f-8f349074983c admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:36.405036 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5344bca8-afd2-4f57-8e1f-8f349074983c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:36.409325 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5344bca8-afd2-4f57-8e1f-8f349074983c admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:36.411254 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a6dc84-639b-403a-840a-3253256f0e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:36.420669 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a6dc84-639b-403a-840a-3253256f0e40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:36.420960 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a6dc84-639b-403a-840a-3253256f0e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:36.421237 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a6dc84-639b-403a-840a-3253256f0e40 admin admin] PATH_INFO: `/v3/projects/027b06570b2a41c4a0b43f3529340077` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:36.428653 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:36.432474 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1282/2576] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:49:36 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:49:36.438643 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5344bca8-afd2-4f57-8e1f-8f349074983c admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:36.445726 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a6dc84-639b-403a-840a-3253256f0e40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=027b06570b2a41c4a0b43f3529340077)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:36.448991 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a6dc84-639b-403a-840a-3253256f0e40 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:36.452338 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5344bca8-afd2-4f57-8e1f-8f349074983c admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:36.458007 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43b4c45-1651-4643-835e-a96ac5ea1511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:36.458831 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b4c45-1651-4643-835e-a96ac5ea1511 admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:36.459176 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b4c45-1651-4643-835e-a96ac5ea1511 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:36.459655 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b4c45-1651-4643-835e-a96ac5ea1511 admin admin] PATH_INFO: `/v3/projects/a42f7b52e8e94476848366d8d3e075fc/users/83f4291247b648289b6ea3b6fe48fdf5/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:36.508079 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43b4c45-1651-4643-835e-a96ac5ea1511 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a42f7b52e8e94476848366d8d3e075fc, user_id=83f4291247b648289b6ea3b6fe48fdf5, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:36.511291 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43b4c45-1651-4643-835e-a96ac5ea1511 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:36.544372 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1283/2577] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:49:36 2022] PUT /identity/v3/projects/a42f7b52e8e94476848366d8d3e075fc/users/83f4291247b648289b6ea3b6fe48fdf5/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:49:36.549845 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b299db8b-63f5-4372-b51a-2054ddfa0202 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:36.550263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b299db8b-63f5-4372-b51a-2054ddfa0202 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:36.550691 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b299db8b-63f5-4372-b51a-2054ddfa0202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:36.589753 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-c8a6dc84-639b-403a-840a-3253256f0e40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '027b06570b2a41c4a0b43f3529340077', 'request_id': 'req-c8a6dc84-639b-403a-840a-3253256f0e40'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:36.603721 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-b299db8b-63f5-4372-b51a-2054ddfa0202 None None] MFA Rules not processed for user `83f4291247b648289b6ea3b6fe48fdf5`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:49:36.605068 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-b299db8b-63f5-4372-b51a-2054ddfa0202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:36.615840 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-c8a6dc84-639b-403a-840a-3253256f0e40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '027b06570b2a41c4a0b43f3529340077', 'request_id': 'req-c8a6dc84-639b-403a-840a-3253256f0e40'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:36.623747 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-c8a6dc84-639b-403a-840a-3253256f0e40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '027b06570b2a41c4a0b43f3529340077', 'request_id': 'req-c8a6dc84-639b-403a-840a-3253256f0e40'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:36.659573 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-c8a6dc84-639b-403a-840a-3253256f0e40 admin admin] The token cache is being invalidate because project 027b06570b2a41c4a0b43f3529340077 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81617) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:49:36.659908 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-c8a6dc84-639b-403a-840a-3253256f0e40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:36.660817 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-c8a6dc84-639b-403a-840a-3253256f0e40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:36.662998 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1295/2578] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:49:36 2022] DELETE /identity/v3/projects/027b06570b2a41c4a0b43f3529340077 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:49:36.668936 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a6dc84-639b-403a-840a-3253256f0e40 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:36.670756 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-c8a6dc84-639b-403a-840a-3253256f0e40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:36.719454 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1284/2579] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:49:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:49:36.723881 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b299db8b-63f5-4372-b51a-2054ddfa0202 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:36.730029 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-b299db8b-63f5-4372-b51a-2054ddfa0202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:36.783836 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a6dc84-639b-403a-840a-3253256f0e40 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:36.792949 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8c934e-30f1-4515-b3e9-bf7bdf7d4853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:36.793638 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c934e-30f1-4515-b3e9-bf7bdf7d4853 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:36.793920 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c934e-30f1-4515-b3e9-bf7bdf7d4853 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:36.795855 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c934e-30f1-4515-b3e9-bf7bdf7d4853 admin admin] PATH_INFO: `/v3/users/edff2b6344e34f1b877d0c908c43290f` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:36.825207 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8c934e-30f1-4515-b3e9-bf7bdf7d4853 admin admin] RBAC: Authorizing `identity:delete_user(user_id=edff2b6344e34f1b877d0c908c43290f)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:36.827248 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8c934e-30f1-4515-b3e9-bf7bdf7d4853 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:36.850176 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:36.864231 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b299db8b-63f5-4372-b51a-2054ddfa0202 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:36.870226 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5344e31f-a41b-49ae-9893-6a6820d57222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:36.871164 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5344e31f-a41b-49ae-9893-6a6820d57222 admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:36.871695 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5344e31f-a41b-49ae-9893-6a6820d57222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:36.872358 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5344e31f-a41b-49ae-9893-6a6820d57222 admin admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:36.873173 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5344e31f-a41b-49ae-9893-6a6820d57222 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:36.876018 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5344e31f-a41b-49ae-9893-6a6820d57222 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:36.940696 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1285/2580] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:49:36 2022] POST /identity/v3/projects => generated 383 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 16:49:36.946954 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5344e31f-a41b-49ae-9893-6a6820d57222 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:36.962341 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-0c8c934e-30f1-4515-b3e9-bf7bdf7d4853 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edff2b6344e34f1b877d0c908c43290f'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:36.963236 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-0c8c934e-30f1-4515-b3e9-bf7bdf7d4853 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edff2b6344e34f1b877d0c908c43290f'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:36.976160 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-0c8c934e-30f1-4515-b3e9-bf7bdf7d4853 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edff2b6344e34f1b877d0c908c43290f'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:36.994410 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-0c8c934e-30f1-4515-b3e9-bf7bdf7d4853 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edff2b6344e34f1b877d0c908c43290f'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:36.995396 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-0c8c934e-30f1-4515-b3e9-bf7bdf7d4853 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edff2b6344e34f1b877d0c908c43290f'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:36.997483 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:37.044217 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1296/2581] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:49:36 2022] DELETE /identity/v3/users/edff2b6344e34f1b877d0c908c43290f => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:49:37.164314 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5344e31f-a41b-49ae-9893-6a6820d57222 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:37.265495 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b0501a-2545-477d-b019-a6dfa2bb2cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:37.266196 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b0501a-2545-477d-b019-a6dfa2bb2cce admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:37.266401 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b0501a-2545-477d-b019-a6dfa2bb2cce admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:37.266675 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b0501a-2545-477d-b019-a6dfa2bb2cce admin admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:37.267324 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b0501a-2545-477d-b019-a6dfa2bb2cce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:37.269315 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b0501a-2545-477d-b019-a6dfa2bb2cce admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:37.303012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1286/2582] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:49:36 2022] POST /identity/v3/users => generated 298 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:49:37.305886 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8c934e-30f1-4515-b3e9-bf7bdf7d4853 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:37.312106 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-0c8c934e-30f1-4515-b3e9-bf7bdf7d4853 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:37.338083 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8c934e-30f1-4515-b3e9-bf7bdf7d4853 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:37.350584 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c621fa0-277a-44cd-9ed9-502917369aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:37.352373 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c621fa0-277a-44cd-9ed9-502917369aac admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:37.353243 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c621fa0-277a-44cd-9ed9-502917369aac admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:37.353681 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c621fa0-277a-44cd-9ed9-502917369aac admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:37.354711 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c621fa0-277a-44cd-9ed9-502917369aac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:37.355586 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b0501a-2545-477d-b019-a6dfa2bb2cce admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:37.358767 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-23b0501a-2545-477d-b019-a6dfa2bb2cce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:37.368566 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c621fa0-277a-44cd-9ed9-502917369aac admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:37.375713 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b0501a-2545-477d-b019-a6dfa2bb2cce admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:37.397196 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1297/2583] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:49:37 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:49:37.401556 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c621fa0-277a-44cd-9ed9-502917369aac admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:37.403338 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcd731e-dd62-4061-a9d3-7a3cf68613f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:37.404007 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd731e-dd62-4061-a9d3-7a3cf68613f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:37.404288 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd731e-dd62-4061-a9d3-7a3cf68613f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:37.404560 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd731e-dd62-4061-a9d3-7a3cf68613f4 admin admin] PATH_INFO: `/v3/projects/ecbaa32c295d45cb83db075c5e180c30` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:37.418443 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c621fa0-277a-44cd-9ed9-502917369aac admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:37.424715 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0138ce74-cff6-43be-acee-37fdb40ef91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:37.425532 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0138ce74-cff6-43be-acee-37fdb40ef91d admin admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:37.425913 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0138ce74-cff6-43be-acee-37fdb40ef91d admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:37.426249 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0138ce74-cff6-43be-acee-37fdb40ef91d admin admin] PATH_INFO: `/v3/projects/ec3c463fd22d45e4abb9f88415db23c5/users/3c79bcbee4d142b1a19efb5b01c2e12f/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:37.443046 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcd731e-dd62-4061-a9d3-7a3cf68613f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecbaa32c295d45cb83db075c5e180c30)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:37.445727 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcd731e-dd62-4061-a9d3-7a3cf68613f4 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:37.454364 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0138ce74-cff6-43be-acee-37fdb40ef91d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec3c463fd22d45e4abb9f88415db23c5, user_id=3c79bcbee4d142b1a19efb5b01c2e12f, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:37.461647 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0138ce74-cff6-43be-acee-37fdb40ef91d admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:37.499723 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1298/2584] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:49:37 2022] PUT /identity/v3/projects/ec3c463fd22d45e4abb9f88415db23c5/users/3c79bcbee4d142b1a19efb5b01c2e12f/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:49:37.504167 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0138ce74-cff6-43be-acee-37fdb40ef91d admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:37.513154 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-9bcd731e-dd62-4061-a9d3-7a3cf68613f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecbaa32c295d45cb83db075c5e180c30', 'request_id': 'req-9bcd731e-dd62-4061-a9d3-7a3cf68613f4'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:37.532908 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-9bcd731e-dd62-4061-a9d3-7a3cf68613f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecbaa32c295d45cb83db075c5e180c30', 'request_id': 'req-9bcd731e-dd62-4061-a9d3-7a3cf68613f4'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:37.545032 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-9bcd731e-dd62-4061-a9d3-7a3cf68613f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecbaa32c295d45cb83db075c5e180c30', 'request_id': 'req-9bcd731e-dd62-4061-a9d3-7a3cf68613f4'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:37.547090 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:DELETE=6 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:37.567765 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-9bcd731e-dd62-4061-a9d3-7a3cf68613f4 admin admin] The token cache is being invalidate because project ecbaa32c295d45cb83db075c5e180c30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81618) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:49:37.567765 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-9bcd731e-dd62-4061-a9d3-7a3cf68613f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:37.568680 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-9bcd731e-dd62-4061-a9d3-7a3cf68613f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:37.571565 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1287/2585] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:49:37 2022] DELETE /identity/v3/projects/ecbaa32c295d45cb83db075c5e180c30 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 16:49:37.651744 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0138ce74-cff6-43be-acee-37fdb40ef91d admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:37.728048 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b5a722-528a-415e-aa95-371e946e3e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:37.728846 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b5a722-528a-415e-aa95-371e946e3e72 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:37.729137 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b5a722-528a-415e-aa95-371e946e3e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:37.729410 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b5a722-528a-415e-aa95-371e946e3e72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:37.730073 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b5a722-528a-415e-aa95-371e946e3e72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:37.731780 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b5a722-528a-415e-aa95-371e946e3e72 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:37.748144 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1299/2586] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:49:37 2022] GET /identity/v3/roles => generated 1420 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:49:37.750425 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcd731e-dd62-4061-a9d3-7a3cf68613f4 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:37.752389 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-9bcd731e-dd62-4061-a9d3-7a3cf68613f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:37.798336 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcd731e-dd62-4061-a9d3-7a3cf68613f4 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:37.810448 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c3635b-7d01-4711-8a98-42103d9c6609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:37.814106 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c3635b-7d01-4711-8a98-42103d9c6609 admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:37.815121 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c3635b-7d01-4711-8a98-42103d9c6609 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:37.817522 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c3635b-7d01-4711-8a98-42103d9c6609 admin admin] PATH_INFO: `/v3/projects/ec3c463fd22d45e4abb9f88415db23c5/users/3c79bcbee4d142b1a19efb5b01c2e12f/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:37.823679 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c3635b-7d01-4711-8a98-42103d9c6609 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec3c463fd22d45e4abb9f88415db23c5, user_id=3c79bcbee4d142b1a19efb5b01c2e12f, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:37.832263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c3635b-7d01-4711-8a98-42103d9c6609 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:37.860532 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1288/2587] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:49:37 2022] PUT /identity/v3/projects/ec3c463fd22d45e4abb9f88415db23c5/users/3c79bcbee4d142b1a19efb5b01c2e12f/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:49:37.872693 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5406e3-8326-428a-bb45-0f686b354726 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:37.872996 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5406e3-8326-428a-bb45-0f686b354726 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:37.874065 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5406e3-8326-428a-bb45-0f686b354726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:37.921759 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-be5406e3-8326-428a-bb45-0f686b354726 None None] MFA Rules not processed for user `3c79bcbee4d142b1a19efb5b01c2e12f`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:49:37.923237 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-be5406e3-8326-428a-bb45-0f686b354726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:38.017899 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:INSERT=2,keystone:DELETE=1,keystone:SELECT=97 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:38.043276 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c3635b-7d01-4711-8a98-42103d9c6609 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:38.047936 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-90c3635b-7d01-4711-8a98-42103d9c6609 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:38.063150 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1300/2588] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:49:37 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:49:38.078539 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5406e3-8326-428a-bb45-0f686b354726 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:38.083070 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-be5406e3-8326-428a-bb45-0f686b354726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:38.153597 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c3635b-7d01-4711-8a98-42103d9c6609 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:38.158330 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5406e3-8326-428a-bb45-0f686b354726 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:38.161857 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a84bbbf-2013-4927-a952-98dbc358ca74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:38.162165 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f213b3-c1d3-4fcf-ae52-9e286d5b54cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:38.162817 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f213b3-c1d3-4fcf-ae52-9e286d5b54cd service cinder] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:38.163092 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f213b3-c1d3-4fcf-ae52-9e286d5b54cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:38.163504 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a84bbbf-2013-4927-a952-98dbc358ca74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:38.164256 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a84bbbf-2013-4927-a952-98dbc358ca74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:38.165001 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a84bbbf-2013-4927-a952-98dbc358ca74 admin admin] PATH_INFO: `/v3/users/772919ee28d54a4b84b8237d04a65879` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:38.166932 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f213b3-c1d3-4fcf-ae52-9e286d5b54cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:38.168542 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a84bbbf-2013-4927-a952-98dbc358ca74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=772919ee28d54a4b84b8237d04a65879)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:38.168765 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-99f213b3-c1d3-4fcf-ae52-9e286d5b54cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:38.170771 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a84bbbf-2013-4927-a952-98dbc358ca74 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:38.245932 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-2a84bbbf-2013-4927-a952-98dbc358ca74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '772919ee28d54a4b84b8237d04a65879'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:38.262022 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f213b3-c1d3-4fcf-ae52-9e286d5b54cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:38.263057 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f213b3-c1d3-4fcf-ae52-9e286d5b54cd service cinder] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:38.263960 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-2a84bbbf-2013-4927-a952-98dbc358ca74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '772919ee28d54a4b84b8237d04a65879'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:38.274640 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1301/2589] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:49:38 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 16:49:38.292426 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-2a84bbbf-2013-4927-a952-98dbc358ca74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '772919ee28d54a4b84b8237d04a65879'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:38.311463 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-2a84bbbf-2013-4927-a952-98dbc358ca74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '772919ee28d54a4b84b8237d04a65879'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:38.312608 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-2a84bbbf-2013-4927-a952-98dbc358ca74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '772919ee28d54a4b84b8237d04a65879'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:38.328620 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1289/2590] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:49:38 2022] DELETE /identity/v3/users/772919ee28d54a4b84b8237d04a65879 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:49:38.333129 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f917b3ba-2076-4b4c-94ae-f48b2cba4505 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:38.333671 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f917b3ba-2076-4b4c-94ae-f48b2cba4505 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:38.333671 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f917b3ba-2076-4b4c-94ae-f48b2cba4505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:38.364349 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-f917b3ba-2076-4b4c-94ae-f48b2cba4505 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:49:38.365613 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-f917b3ba-2076-4b4c-94ae-f48b2cba4505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:38.485264 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1302/2591] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:49:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:49:38.498023 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a84bbbf-2013-4927-a952-98dbc358ca74 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:38.499891 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-2a84bbbf-2013-4927-a952-98dbc358ca74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:38.597687 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a84bbbf-2013-4927-a952-98dbc358ca74 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:38.603533 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bca9d02-9f5e-434e-94b9-62d26f1c9b95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:38.604250 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bca9d02-9f5e-434e-94b9-62d26f1c9b95 service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:38.604529 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bca9d02-9f5e-434e-94b9-62d26f1c9b95 service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:38.604934 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bca9d02-9f5e-434e-94b9-62d26f1c9b95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:38.621715 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bca9d02-9f5e-434e-94b9-62d26f1c9b95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:38.624505 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bca9d02-9f5e-434e-94b9-62d26f1c9b95 service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:38.630965 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:INSERT=3,keystone:DELETE=11,keystone:SELECT=86 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:38.640227 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1290/2592] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:49:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:49:39.025380 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f917b3ba-2076-4b4c-94ae-f48b2cba4505 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:39.027374 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-f917b3ba-2076-4b4c-94ae-f48b2cba4505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:39.051423 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f917b3ba-2076-4b4c-94ae-f48b2cba4505 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:39.071026 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90f9093-014c-4db5-89b2-8da3ae5da895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:39.071752 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f9093-014c-4db5-89b2-8da3ae5da895 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:39.072028 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f9093-014c-4db5-89b2-8da3ae5da895 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:39.072773 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f9093-014c-4db5-89b2-8da3ae5da895 admin admin] PATH_INFO: `/v3/projects/f88505764705431cbea1bb555f198999` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:39.074493 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90f9093-014c-4db5-89b2-8da3ae5da895 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f88505764705431cbea1bb555f198999)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:39.077657 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90f9093-014c-4db5-89b2-8da3ae5da895 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:39.183491 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:39.216535 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-c90f9093-014c-4db5-89b2-8da3ae5da895 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f88505764705431cbea1bb555f198999', 'request_id': 'req-c90f9093-014c-4db5-89b2-8da3ae5da895'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:39.225438 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-c90f9093-014c-4db5-89b2-8da3ae5da895 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f88505764705431cbea1bb555f198999', 'request_id': 'req-c90f9093-014c-4db5-89b2-8da3ae5da895'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:39.248802 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-c90f9093-014c-4db5-89b2-8da3ae5da895 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f88505764705431cbea1bb555f198999', 'request_id': 'req-c90f9093-014c-4db5-89b2-8da3ae5da895'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:39.278186 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-c90f9093-014c-4db5-89b2-8da3ae5da895 admin admin] The token cache is being invalidate because project f88505764705431cbea1bb555f198999 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81617) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:49:39.278186 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-c90f9093-014c-4db5-89b2-8da3ae5da895 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:39.280202 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-c90f9093-014c-4db5-89b2-8da3ae5da895 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:39.286481 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1303/2593] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:49:39 2022] DELETE /identity/v3/projects/f88505764705431cbea1bb555f198999 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:49:39.286776 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bca9d02-9f5e-434e-94b9-62d26f1c9b95 service neutron] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:39.309221 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bca9d02-9f5e-434e-94b9-62d26f1c9b95 service neutron] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:39.384279 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a371f7fe-52f1-4b38-a571-7132443c12aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:39.385107 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a371f7fe-52f1-4b38-a571-7132443c12aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:39.385463 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a371f7fe-52f1-4b38-a571-7132443c12aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:39.385742 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a371f7fe-52f1-4b38-a571-7132443c12aa admin admin] PATH_INFO: `/v3/users/837c0a783a7e4315a71dad78757ae92d` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:39.386964 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a371f7fe-52f1-4b38-a571-7132443c12aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=837c0a783a7e4315a71dad78757ae92d)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:39.388883 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a371f7fe-52f1-4b38-a571-7132443c12aa admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:39.466742 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-a371f7fe-52f1-4b38-a571-7132443c12aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '837c0a783a7e4315a71dad78757ae92d'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:39.475011 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-a371f7fe-52f1-4b38-a571-7132443c12aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '837c0a783a7e4315a71dad78757ae92d'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:39.491213 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-a371f7fe-52f1-4b38-a571-7132443c12aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '837c0a783a7e4315a71dad78757ae92d'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:39.503304 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-a371f7fe-52f1-4b38-a571-7132443c12aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '837c0a783a7e4315a71dad78757ae92d'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:39.503989 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-a371f7fe-52f1-4b38-a571-7132443c12aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '837c0a783a7e4315a71dad78757ae92d'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:39.518113 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1291/2594] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:49:39 2022] DELETE /identity/v3/users/837c0a783a7e4315a71dad78757ae92d => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 16:49:39.912981 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90f9093-014c-4db5-89b2-8da3ae5da895 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:39.916010 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-c90f9093-014c-4db5-89b2-8da3ae5da895 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:39.948242 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90f9093-014c-4db5-89b2-8da3ae5da895 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:40.057416 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d0bdd3-b9e5-4c1d-999f-6e3e626cc010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:40.059241 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d0bdd3-b9e5-4c1d-999f-6e3e626cc010 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:40.060027 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d0bdd3-b9e5-4c1d-999f-6e3e626cc010 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:40.061092 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d0bdd3-b9e5-4c1d-999f-6e3e626cc010 admin admin] PATH_INFO: `/v3/projects/b8e7160e42444ea8ace7e7a0fe159e55` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:40.064483 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d0bdd3-b9e5-4c1d-999f-6e3e626cc010 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8e7160e42444ea8ace7e7a0fe159e55)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:40.069116 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d0bdd3-b9e5-4c1d-999f-6e3e626cc010 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:40.110251 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a371f7fe-52f1-4b38-a571-7132443c12aa admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:40.112014 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-a371f7fe-52f1-4b38-a571-7132443c12aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:40.134290 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-61d0bdd3-b9e5-4c1d-999f-6e3e626cc010 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8e7160e42444ea8ace7e7a0fe159e55', 'request_id': 'req-61d0bdd3-b9e5-4c1d-999f-6e3e626cc010'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:40.144331 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-61d0bdd3-b9e5-4c1d-999f-6e3e626cc010 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8e7160e42444ea8ace7e7a0fe159e55', 'request_id': 'req-61d0bdd3-b9e5-4c1d-999f-6e3e626cc010'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:40.155214 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-61d0bdd3-b9e5-4c1d-999f-6e3e626cc010 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8e7160e42444ea8ace7e7a0fe159e55', 'request_id': 'req-61d0bdd3-b9e5-4c1d-999f-6e3e626cc010'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:40.185390 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-61d0bdd3-b9e5-4c1d-999f-6e3e626cc010 admin admin] The token cache is being invalidate because project b8e7160e42444ea8ace7e7a0fe159e55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81617) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:49:40.185390 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-61d0bdd3-b9e5-4c1d-999f-6e3e626cc010 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:40.185390 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-61d0bdd3-b9e5-4c1d-999f-6e3e626cc010 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:40.185390 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1304/2595] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:49:39 2022] DELETE /identity/v3/projects/b8e7160e42444ea8ace7e7a0fe159e55 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:49:40.185806 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1d40f-1835-4a6c-8ca9-9d39cde4dd7b None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:40.186201 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1d40f-1835-4a6c-8ca9-9d39cde4dd7b None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:40.186588 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1d40f-1835-4a6c-8ca9-9d39cde4dd7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:40.208529 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a371f7fe-52f1-4b38-a571-7132443c12aa admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:40.217491 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-76b1d40f-1835-4a6c-8ca9-9d39cde4dd7b None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:49:40.218835 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-76b1d40f-1835-4a6c-8ca9-9d39cde4dd7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:40.250288 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:40.270506 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c19f412-eb32-42b5-9e95-1a481a0acd05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:40.271133 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c19f412-eb32-42b5-9e95-1a481a0acd05 service cinder] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:40.271409 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c19f412-eb32-42b5-9e95-1a481a0acd05 service cinder] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:40.271678 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c19f412-eb32-42b5-9e95-1a481a0acd05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:40.273152 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:40.273937 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-2c19f412-eb32-42b5-9e95-1a481a0acd05 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:40.287481 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1305/2596] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:49:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:49:40.296119 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbe0c22-dfbb-4daa-a8d7-617fb71dac1c None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:40.296395 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbe0c22-dfbb-4daa-a8d7-617fb71dac1c None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:40.296669 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbe0c22-dfbb-4daa-a8d7-617fb71dac1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:40.319956 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-cbbe0c22-dfbb-4daa-a8d7-617fb71dac1c None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:49:40.321279 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-cbbe0c22-dfbb-4daa-a8d7-617fb71dac1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:40.331838 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1306/2597] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:49:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:49:40.336343 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbe0c22-dfbb-4daa-a8d7-617fb71dac1c None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:40.340789 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c19f412-eb32-42b5-9e95-1a481a0acd05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:40.342258 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c19f412-eb32-42b5-9e95-1a481a0acd05 service cinder] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:40.347681 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbe0c22-dfbb-4daa-a8d7-617fb71dac1c None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:40.353373 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1292/2598] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:49:40 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 16:49:40.353595 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a795a52-71fd-4c72-a6ba-f879db17fb87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:40.353970 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a795a52-71fd-4c72-a6ba-f879db17fb87 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:40.354249 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a795a52-71fd-4c72-a6ba-f879db17fb87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:40.354550 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a795a52-71fd-4c72-a6ba-f879db17fb87 admin admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:40.355214 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a795a52-71fd-4c72-a6ba-f879db17fb87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:49:40.355671 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a795a52-71fd-4c72-a6ba-f879db17fb87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:40.357288 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a795a52-71fd-4c72-a6ba-f879db17fb87 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:40.371163 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1307/2599] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:49:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:49:40.376677 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c19f412-eb32-42b5-9e95-1a481a0acd05 service cinder] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:40.388389 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c19f412-eb32-42b5-9e95-1a481a0acd05 service cinder] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:40.393207 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790d8eaa-4489-4c45-a6d7-ecbc7e76f4a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:40.393906 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d8eaa-4489-4c45-a6d7-ecbc7e76f4a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:40.394258 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d8eaa-4489-4c45-a6d7-ecbc7e76f4a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:40.394687 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d8eaa-4489-4c45-a6d7-ecbc7e76f4a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:40.395514 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790d8eaa-4489-4c45-a6d7-ecbc7e76f4a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:40.398702 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790d8eaa-4489-4c45-a6d7-ecbc7e76f4a3 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:40.433450 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1293/2600] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:49:40 2022] POST /identity/v3/projects => generated 409 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 16:49:40.434713 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a795a52-71fd-4c72-a6ba-f879db17fb87 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:40.446957 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a795a52-71fd-4c72-a6ba-f879db17fb87 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:40.528659 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd1cb1d-c124-4c4e-9cb7-7014fcc4c915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:40.530238 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd1cb1d-c124-4c4e-9cb7-7014fcc4c915 admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:40.530812 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd1cb1d-c124-4c4e-9cb7-7014fcc4c915 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:40.531437 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd1cb1d-c124-4c4e-9cb7-7014fcc4c915 admin admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:40.532864 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd1cb1d-c124-4c4e-9cb7-7014fcc4c915 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:40.536627 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd1cb1d-c124-4c4e-9cb7-7014fcc4c915 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:40.575691 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1308/2601] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:49:40 2022] POST /identity/v3/users => generated 305 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:49:40.580242 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790d8eaa-4489-4c45-a6d7-ecbc7e76f4a3 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:40.598298 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790d8eaa-4489-4c45-a6d7-ecbc7e76f4a3 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:40.607169 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8452d377-a70d-4853-9d0d-07211a2a9aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:40.608166 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8452d377-a70d-4853-9d0d-07211a2a9aa7 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:40.608678 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8452d377-a70d-4853-9d0d-07211a2a9aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:40.609185 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8452d377-a70d-4853-9d0d-07211a2a9aa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:40.610412 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8452d377-a70d-4853-9d0d-07211a2a9aa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:40.613499 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8452d377-a70d-4853-9d0d-07211a2a9aa7 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:40.634682 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1294/2602] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:49:40 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:49:40.639488 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd1cb1d-c124-4c4e-9cb7-7014fcc4c915 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:40.652278 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd1cb1d-c124-4c4e-9cb7-7014fcc4c915 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:40.660210 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e682d6-bd81-439d-9fe6-c2fd37af0f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:40.661464 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e682d6-bd81-439d-9fe6-c2fd37af0f60 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:40.661993 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e682d6-bd81-439d-9fe6-c2fd37af0f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:40.662562 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e682d6-bd81-439d-9fe6-c2fd37af0f60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:40.663833 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e682d6-bd81-439d-9fe6-c2fd37af0f60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:40.665825 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e682d6-bd81-439d-9fe6-c2fd37af0f60 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:40.693901 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1309/2603] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:49:40 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:49:40.698013 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8452d377-a70d-4853-9d0d-07211a2a9aa7 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:40.709426 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8452d377-a70d-4853-9d0d-07211a2a9aa7 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:40.714954 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce662fd-2952-4020-a941-45c25bb7862c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:40.715690 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce662fd-2952-4020-a941-45c25bb7862c admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:40.715958 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce662fd-2952-4020-a941-45c25bb7862c admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:40.716234 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce662fd-2952-4020-a941-45c25bb7862c admin admin] PATH_INFO: `/v3/projects/223a5e788741496c9e98a91cde6aadbc/users/91df23aa51134c29aa4f09640fa87b33/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:40.732205 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce662fd-2952-4020-a941-45c25bb7862c admin admin] RBAC: Authorizing `identity:create_grant(project_id=223a5e788741496c9e98a91cde6aadbc, user_id=91df23aa51134c29aa4f09640fa87b33, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:40.734886 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce662fd-2952-4020-a941-45c25bb7862c admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:40.753627 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1295/2604] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:49:40 2022] PUT /identity/v3/projects/223a5e788741496c9e98a91cde6aadbc/users/91df23aa51134c29aa4f09640fa87b33/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:49:40.765759 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc5006-caba-4c5b-a223-6376c597424a None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:40.766043 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc5006-caba-4c5b-a223-6376c597424a None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:40.766328 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc5006-caba-4c5b-a223-6376c597424a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:40.798432 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-8cbc5006-caba-4c5b-a223-6376c597424a None None] MFA Rules not processed for user `91df23aa51134c29aa4f09640fa87b33`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:49:40.799687 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-8cbc5006-caba-4c5b-a223-6376c597424a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:40.880733 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1310/2605] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:49:40 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:49:40.886248 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce662fd-2952-4020-a941-45c25bb7862c admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:40.957967 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce662fd-2952-4020-a941-45c25bb7862c admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:40.962863 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f598a39-eae3-4298-8772-be56f02968bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:40.963478 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f598a39-eae3-4298-8772-be56f02968bd admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:40.963759 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f598a39-eae3-4298-8772-be56f02968bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:40.964033 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f598a39-eae3-4298-8772-be56f02968bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:40.964750 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f598a39-eae3-4298-8772-be56f02968bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:40.966748 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f598a39-eae3-4298-8772-be56f02968bd admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:40.998585 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1296/2606] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:49:40 2022] POST /identity/v3/projects => generated 407 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 16:49:40.999680 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbc5006-caba-4c5b-a223-6376c597424a None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:41.011922 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbc5006-caba-4c5b-a223-6376c597424a None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:41.063045 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:41.082793 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d4ff8b-b237-45c3-8bf3-88953e30ecd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:41.083415 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d4ff8b-b237-45c3-8bf3-88953e30ecd6 admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:41.083697 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d4ff8b-b237-45c3-8bf3-88953e30ecd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:41.083973 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d4ff8b-b237-45c3-8bf3-88953e30ecd6 admin admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:41.084630 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d4ff8b-b237-45c3-8bf3-88953e30ecd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:41.086634 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d4ff8b-b237-45c3-8bf3-88953e30ecd6 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:41.109607 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1311/2607] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:49:40 2022] POST /identity/v3/users => generated 310 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:49:41.115486 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f598a39-eae3-4298-8772-be56f02968bd admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:41.127363 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f598a39-eae3-4298-8772-be56f02968bd admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:41.132238 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5e916a-3e62-4d0c-841e-fe82a42c1929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:41.132859 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5e916a-3e62-4d0c-841e-fe82a42c1929 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:41.133141 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5e916a-3e62-4d0c-841e-fe82a42c1929 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:41.133425 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5e916a-3e62-4d0c-841e-fe82a42c1929 admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:41.134069 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5e916a-3e62-4d0c-841e-fe82a42c1929 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:41.135712 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5e916a-3e62-4d0c-841e-fe82a42c1929 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:41.147222 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1297/2608] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:49:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:49:41.151735 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d4ff8b-b237-45c3-8bf3-88953e30ecd6 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:41.162204 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d4ff8b-b237-45c3-8bf3-88953e30ecd6 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:41.167572 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b539ad18-9e8a-4acb-aca5-24266305aac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:41.168247 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b539ad18-9e8a-4acb-aca5-24266305aac1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:41.168528 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b539ad18-9e8a-4acb-aca5-24266305aac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:41.168804 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b539ad18-9e8a-4acb-aca5-24266305aac1 admin admin] PATH_INFO: `/v3/projects/21d5dc352dfa45b7865125b8137fae28/users/765ec0dfea0441dd952ea8a1ef693a5b/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:41.186066 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b539ad18-9e8a-4acb-aca5-24266305aac1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21d5dc352dfa45b7865125b8137fae28, user_id=765ec0dfea0441dd952ea8a1ef693a5b, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:41.188837 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b539ad18-9e8a-4acb-aca5-24266305aac1 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:41.204488 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1312/2609] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:49:41 2022] PUT /identity/v3/projects/21d5dc352dfa45b7865125b8137fae28/users/765ec0dfea0441dd952ea8a1ef693a5b/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:49:41.210451 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5e916a-3e62-4d0c-841e-fe82a42c1929 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:41.224389 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5e916a-3e62-4d0c-841e-fe82a42c1929 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:41.231140 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:41.295783 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67364ebc-9173-4d98-96a3-c68cf3a8f2d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:41.296408 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67364ebc-9173-4d98-96a3-c68cf3a8f2d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:41.296689 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67364ebc-9173-4d98-96a3-c68cf3a8f2d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:41.296963 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67364ebc-9173-4d98-96a3-c68cf3a8f2d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:41.297605 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67364ebc-9173-4d98-96a3-c68cf3a8f2d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:41.299223 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67364ebc-9173-4d98-96a3-c68cf3a8f2d8 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:41.310579 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1298/2610] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:49:41 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:49:41.314938 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b539ad18-9e8a-4acb-aca5-24266305aac1 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:41.326760 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b539ad18-9e8a-4acb-aca5-24266305aac1 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:41.333028 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56a3e06-dcec-4264-ba4c-b1b8dc16196e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:41.333703 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56a3e06-dcec-4264-ba4c-b1b8dc16196e admin admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:41.333983 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56a3e06-dcec-4264-ba4c-b1b8dc16196e admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:41.334262 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56a3e06-dcec-4264-ba4c-b1b8dc16196e admin admin] PATH_INFO: `/v3/projects/21d5dc352dfa45b7865125b8137fae28/users/765ec0dfea0441dd952ea8a1ef693a5b/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:41.336349 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56a3e06-dcec-4264-ba4c-b1b8dc16196e admin admin] RBAC: Authorizing `identity:create_grant(project_id=21d5dc352dfa45b7865125b8137fae28, user_id=765ec0dfea0441dd952ea8a1ef693a5b, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:41.339212 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56a3e06-dcec-4264-ba4c-b1b8dc16196e admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:41.360005 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1313/2611] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:49:41 2022] PUT /identity/v3/projects/21d5dc352dfa45b7865125b8137fae28/users/765ec0dfea0441dd952ea8a1ef693a5b/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:49:41.371436 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e874aee-5258-4994-8f75-9a177b4d4ec1 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:41.372228 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e874aee-5258-4994-8f75-9a177b4d4ec1 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:41.372228 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e874aee-5258-4994-8f75-9a177b4d4ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:41.417716 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-3e874aee-5258-4994-8f75-9a177b4d4ec1 None None] MFA Rules not processed for user `765ec0dfea0441dd952ea8a1ef693a5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:49:41.419759 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-3e874aee-5258-4994-8f75-9a177b4d4ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:41.516131 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1299/2612] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:49:41 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:49:41.527199 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56a3e06-dcec-4264-ba4c-b1b8dc16196e admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:41.529160 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-b56a3e06-dcec-4264-ba4c-b1b8dc16196e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:41.597190 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56a3e06-dcec-4264-ba4c-b1b8dc16196e admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:41.601709 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4f4be7-2ac6-4678-9655-53762824d8d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:41.602244 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f4be7-2ac6-4678-9655-53762824d8d5 service cinder] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:41.602554 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f4be7-2ac6-4678-9655-53762824d8d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:41.602827 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f4be7-2ac6-4678-9655-53762824d8d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:41.619718 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4f4be7-2ac6-4678-9655-53762824d8d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:41.621158 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4f4be7-2ac6-4678-9655-53762824d8d5 service cinder] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:41.631467 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1314/2613] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:49:41 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:49:41.749749 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e874aee-5258-4994-8f75-9a177b4d4ec1 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:41.752038 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-3e874aee-5258-4994-8f75-9a177b4d4ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:41.776794 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e874aee-5258-4994-8f75-9a177b4d4ec1 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:41.784595 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab4bb8c-41e2-4514-a15c-fb472f85f977 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:41.785766 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab4bb8c-41e2-4514-a15c-fb472f85f977 service cinder] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:41.786215 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab4bb8c-41e2-4514-a15c-fb472f85f977 service cinder] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:41.786688 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab4bb8c-41e2-4514-a15c-fb472f85f977 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:41.790340 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-9ab4bb8c-41e2-4514-a15c-fb472f85f977 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:41.831414 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4f4be7-2ac6-4678-9655-53762824d8d5 service cinder] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:41.840466 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4f4be7-2ac6-4678-9655-53762824d8d5 service cinder] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:41.844818 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bca47f6-e2d8-435d-872e-6c6b1a60ef7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:41.845375 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca47f6-e2d8-435d-872e-6c6b1a60ef7e service cinder] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:41.845648 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca47f6-e2d8-435d-872e-6c6b1a60ef7e service cinder] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:41.845925 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca47f6-e2d8-435d-872e-6c6b1a60ef7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:41.878269 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab4bb8c-41e2-4514-a15c-fb472f85f977 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:41.879738 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab4bb8c-41e2-4514-a15c-fb472f85f977 service cinder] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:41.890616 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1300/2614] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:49:41 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:49:41.926278 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bca47f6-e2d8-435d-872e-6c6b1a60ef7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:41.927755 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bca47f6-e2d8-435d-872e-6c6b1a60ef7e service cinder] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:41.940588 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1315/2615] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:49:41 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:49:47.011704 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0302c987-2121-4b7c-a415-d802540c2130 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:47.012282 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0302c987-2121-4b7c-a415-d802540c2130 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:47.012282 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0302c987-2121-4b7c-a415-d802540c2130 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:47.013992 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1301/2616] 10.222.0.57 () {54 vars in 921 bytes} [Mon Sep 19 16:49:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:49:47.019728 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe243e-0528-4027-a1e8-c77f1899b4c9 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:47.020229 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe243e-0528-4027-a1e8-c77f1899b4c9 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:47.020416 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe243e-0528-4027-a1e8-c77f1899b4c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:47.061840 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-ffbe243e-0528-4027-a1e8-c77f1899b4c9 None None] MFA Rules not processed for user `1ecd01cc72834b72b5ff2c4b47249737`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:49:47.063237 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-ffbe243e-0528-4027-a1e8-c77f1899b4c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:47.081856 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:47.145235 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1316/2617] 10.222.0.57 () {58 vars in 1018 bytes} [Mon Sep 19 16:49:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:49:47.158449 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0302c987-2121-4b7c-a415-d802540c2130 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:47.160279 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-0302c987-2121-4b7c-a415-d802540c2130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:47.187545 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:47.242985 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0302c987-2121-4b7c-a415-d802540c2130 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:47.246683 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4276c69b-13e0-4885-9037-83c28909030d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5632ade0cc6f4ae28f72160de68486cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:47.247295 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4276c69b-13e0-4885-9037-83c28909030d service glance] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:47.247590 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4276c69b-13e0-4885-9037-83c28909030d service glance] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:47.247847 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4276c69b-13e0-4885-9037-83c28909030d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:47.259084 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4276c69b-13e0-4885-9037-83c28909030d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:47.260511 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4276c69b-13e0-4885-9037-83c28909030d service glance] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:47.271910 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1302/2618] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:49:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:49:47.355040 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23b80c-6543-46a5-92ae-af30512bb016 None None] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:47.355310 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23b80c-6543-46a5-92ae-af30512bb016 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:47.355590 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23b80c-6543-46a5-92ae-af30512bb016 None None] PATH_INFO: `/` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:47.356940 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1317/2619] 10.222.0.57 () {54 vars in 921 bytes} [Mon Sep 19 16:49:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:49:47.361507 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2608ee7-dfd8-435f-b476-a17cbf576fba None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:47.361795 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2608ee7-dfd8-435f-b476-a17cbf576fba None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:47.362072 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2608ee7-dfd8-435f-b476-a17cbf576fba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:47.400246 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-e2608ee7-dfd8-435f-b476-a17cbf576fba None None] MFA Rules not processed for user `50355d7b1c6c4f0bbbbc7fe56efb9039`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:49:47.401458 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-e2608ee7-dfd8-435f-b476-a17cbf576fba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:47.452065 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1303/2620] 10.222.0.57 () {58 vars in 1018 bytes} [Mon Sep 19 16:49:47 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:49:47.464714 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e23b80c-6543-46a5-92ae-af30512bb016 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:47.466481 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-2e23b80c-6543-46a5-92ae-af30512bb016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:47.479576 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e23b80c-6543-46a5-92ae-af30512bb016 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:47.483238 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dac2354-3708-4555-8695-639383ad8732 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:47.483827 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac2354-3708-4555-8695-639383ad8732 service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:47.484107 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac2354-3708-4555-8695-639383ad8732 service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:47.484380 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac2354-3708-4555-8695-639383ad8732 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:47.491815 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dac2354-3708-4555-8695-639383ad8732 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:47.493228 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dac2354-3708-4555-8695-639383ad8732 service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:47.500836 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1318/2621] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:49:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:49:47.586823 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a7b274-cb6f-4a46-8cb2-b46168834322 None None] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:47.587109 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a7b274-cb6f-4a46-8cb2-b46168834322 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:47.587381 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a7b274-cb6f-4a46-8cb2-b46168834322 None None] PATH_INFO: `/` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:47.588754 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1304/2622] 10.222.0.57 () {54 vars in 921 bytes} [Mon Sep 19 16:49:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 16:49:47.593658 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166814a1-de43-4752-b407-0aeb7b72a6bc None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:47.593951 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166814a1-de43-4752-b407-0aeb7b72a6bc None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:47.594229 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166814a1-de43-4752-b407-0aeb7b72a6bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:47.614946 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-166814a1-de43-4752-b407-0aeb7b72a6bc None None] MFA Rules not processed for user `1ecd01cc72834b72b5ff2c4b47249737`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:49:47.616190 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-166814a1-de43-4752-b407-0aeb7b72a6bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:47.626499 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1319/2623] 10.222.0.57 () {58 vars in 1018 bytes} [Mon Sep 19 16:49:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:49:47.639197 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a7b274-cb6f-4a46-8cb2-b46168834322 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:47.650833 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a7b274-cb6f-4a46-8cb2-b46168834322 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:47.654652 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154d66bd-2a6c-4313-9b9d-8c2ad6d4e740 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:47.655250 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d66bd-2a6c-4313-9b9d-8c2ad6d4e740 service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:47.655527 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d66bd-2a6c-4313-9b9d-8c2ad6d4e740 service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:47.655818 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d66bd-2a6c-4313-9b9d-8c2ad6d4e740 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:47.666723 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154d66bd-2a6c-4313-9b9d-8c2ad6d4e740 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:47.668135 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154d66bd-2a6c-4313-9b9d-8c2ad6d4e740 service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:47.679066 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1305/2624] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:49:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 16:49:51.250350 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166814a1-de43-4752-b407-0aeb7b72a6bc None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:51.253964 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-166814a1-de43-4752-b407-0aeb7b72a6bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:51.344093 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166814a1-de43-4752-b407-0aeb7b72a6bc None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:51.349354 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed2fe4e-0958-42d0-9598-7a01047f18f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:51.350011 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed2fe4e-0958-42d0-9598-7a01047f18f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:51.350290 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed2fe4e-0958-42d0-9598-7a01047f18f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:51.350622 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed2fe4e-0958-42d0-9598-7a01047f18f1 admin admin] PATH_INFO: `/v3/users/59d27a486fa54430aa97694b65f69e36` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:51.351893 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed2fe4e-0958-42d0-9598-7a01047f18f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59d27a486fa54430aa97694b65f69e36)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:51.353841 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed2fe4e-0958-42d0-9598-7a01047f18f1 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:51.414456 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-0ed2fe4e-0958-42d0-9598-7a01047f18f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59d27a486fa54430aa97694b65f69e36'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:51.415147 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-0ed2fe4e-0958-42d0-9598-7a01047f18f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59d27a486fa54430aa97694b65f69e36'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:51.434758 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-0ed2fe4e-0958-42d0-9598-7a01047f18f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59d27a486fa54430aa97694b65f69e36'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:51.454003 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-0ed2fe4e-0958-42d0-9598-7a01047f18f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59d27a486fa54430aa97694b65f69e36'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:51.454657 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-0ed2fe4e-0958-42d0-9598-7a01047f18f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59d27a486fa54430aa97694b65f69e36'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:51.475422 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1320/2625] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:49:51 2022] DELETE /identity/v3/users/59d27a486fa54430aa97694b65f69e36 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:49:51.477137 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28e0c59-66a9-4ad4-83b2-40224b3830d8 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:51.477651 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28e0c59-66a9-4ad4-83b2-40224b3830d8 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:51.477651 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28e0c59-66a9-4ad4-83b2-40224b3830d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:51.497471 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-a28e0c59-66a9-4ad4-83b2-40224b3830d8 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:49:51.502585 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-a28e0c59-66a9-4ad4-83b2-40224b3830d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:51.580357 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1306/2626] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:49:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:49:51.587616 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed2fe4e-0958-42d0-9598-7a01047f18f1 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:51.589096 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-0ed2fe4e-0958-42d0-9598-7a01047f18f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:51.605850 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:51.654879 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed2fe4e-0958-42d0-9598-7a01047f18f1 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:51.658890 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e194ac6a-a043-45bf-9953-3046975da3e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:51.659556 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194ac6a-a043-45bf-9953-3046975da3e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:51.659833 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194ac6a-a043-45bf-9953-3046975da3e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:51.660265 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194ac6a-a043-45bf-9953-3046975da3e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:51.672720 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e194ac6a-a043-45bf-9953-3046975da3e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:51.674485 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e194ac6a-a043-45bf-9953-3046975da3e0 service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:51.688958 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1321/2627] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:49:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:49:52.056233 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0951e5f0-40f6-4988-9c26-3d691cdcaae8 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:52.056233 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0951e5f0-40f6-4988-9c26-3d691cdcaae8 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:52.056688 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0951e5f0-40f6-4988-9c26-3d691cdcaae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:52.084187 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-0951e5f0-40f6-4988-9c26-3d691cdcaae8 None None] MFA Rules not processed for user `7222bd3b313b47e5a4fa0ebc9ff0cdd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:49:52.086969 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-0951e5f0-40f6-4988-9c26-3d691cdcaae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:52.099587 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e194ac6a-a043-45bf-9953-3046975da3e0 service neutron] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:52.101044 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-e194ac6a-a043-45bf-9953-3046975da3e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:52.123782 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e194ac6a-a043-45bf-9953-3046975da3e0 service neutron] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:52.130396 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad47eeb-ea60-4918-830b-004bb92325be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:52.131153 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad47eeb-ea60-4918-830b-004bb92325be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:52.131389 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad47eeb-ea60-4918-830b-004bb92325be admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:52.131627 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad47eeb-ea60-4918-830b-004bb92325be admin admin] PATH_INFO: `/v3/projects/e2e1e5be5977467fb5f7946d8c0729b1` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:52.133039 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad47eeb-ea60-4918-830b-004bb92325be admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2e1e5be5977467fb5f7946d8c0729b1)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:52.133847 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:52.134917 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad47eeb-ea60-4918-830b-004bb92325be admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:52.174724 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1307/2628] 10.222.0.57 () {58 vars in 1009 bytes} [Mon Sep 19 16:49:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:49:52.185459 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0951e5f0-40f6-4988-9c26-3d691cdcaae8 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:52.186998 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-0951e5f0-40f6-4988-9c26-3d691cdcaae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:52.199567 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-4ad47eeb-ea60-4918-830b-004bb92325be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2e1e5be5977467fb5f7946d8c0729b1', 'request_id': 'req-4ad47eeb-ea60-4918-830b-004bb92325be'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:52.212503 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-4ad47eeb-ea60-4918-830b-004bb92325be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2e1e5be5977467fb5f7946d8c0729b1', 'request_id': 'req-4ad47eeb-ea60-4918-830b-004bb92325be'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:52.222750 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-4ad47eeb-ea60-4918-830b-004bb92325be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2e1e5be5977467fb5f7946d8c0729b1', 'request_id': 'req-4ad47eeb-ea60-4918-830b-004bb92325be'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:52.243954 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-4ad47eeb-ea60-4918-830b-004bb92325be admin admin] The token cache is being invalidate because project e2e1e5be5977467fb5f7946d8c0729b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81617) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:49:52.244437 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-4ad47eeb-ea60-4918-830b-004bb92325be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:52.245420 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-4ad47eeb-ea60-4918-830b-004bb92325be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:52.247839 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1322/2629] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:49:52 2022] DELETE /identity/v3/projects/e2e1e5be5977467fb5f7946d8c0729b1 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:49:52.251998 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad47eeb-ea60-4918-830b-004bb92325be admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:52.253508 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-4ad47eeb-ea60-4918-830b-004bb92325be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:52.281249 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0951e5f0-40f6-4988-9c26-3d691cdcaae8 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:52.350252 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6abd5a-12cc-4cba-9a7e-868ab14e7e20 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '884a0524aca744b99c7496c11c3291ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:52.350893 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6abd5a-12cc-4cba-9a7e-868ab14e7e20 service swift] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:52.351176 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6abd5a-12cc-4cba-9a7e-868ab14e7e20 service swift] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:52.351495 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6abd5a-12cc-4cba-9a7e-868ab14e7e20 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:52.353395 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-9d6abd5a-12cc-4cba-9a7e-868ab14e7e20 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:52.358392 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad47eeb-ea60-4918-830b-004bb92325be admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:52.364500 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136ba814-6df1-4720-aaeb-c131facac8bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:52.365167 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136ba814-6df1-4720-aaeb-c131facac8bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:52.365483 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136ba814-6df1-4720-aaeb-c131facac8bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:52.365785 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136ba814-6df1-4720-aaeb-c131facac8bd admin admin] PATH_INFO: `/v3/users/83f4291247b648289b6ea3b6fe48fdf5` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:52.366916 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136ba814-6df1-4720-aaeb-c131facac8bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=83f4291247b648289b6ea3b6fe48fdf5)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:52.368871 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136ba814-6df1-4720-aaeb-c131facac8bd admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:52.416899 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6abd5a-12cc-4cba-9a7e-868ab14e7e20 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:52.418481 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:52.418656 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6abd5a-12cc-4cba-9a7e-868ab14e7e20 service swift] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:52.425750 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-136ba814-6df1-4720-aaeb-c131facac8bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83f4291247b648289b6ea3b6fe48fdf5'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:52.426690 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-136ba814-6df1-4720-aaeb-c131facac8bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83f4291247b648289b6ea3b6fe48fdf5'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:52.428660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1308/2630] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Sep 19 16:49:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 19 16:49:52.438105 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-136ba814-6df1-4720-aaeb-c131facac8bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83f4291247b648289b6ea3b6fe48fdf5'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:52.454917 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-136ba814-6df1-4720-aaeb-c131facac8bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83f4291247b648289b6ea3b6fe48fdf5'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:52.455869 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-136ba814-6df1-4720-aaeb-c131facac8bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83f4291247b648289b6ea3b6fe48fdf5'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:52.472593 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1323/2631] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:49:52 2022] DELETE /identity/v3/users/83f4291247b648289b6ea3b6fe48fdf5 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 16:49:52.936483 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6abd5a-12cc-4cba-9a7e-868ab14e7e20 service swift] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:52.951374 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6abd5a-12cc-4cba-9a7e-868ab14e7e20 service swift] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:52.997222 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=100 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:53.094110 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7fb1a6-a109-4abd-88ef-bbbec1c775db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:53.094795 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7fb1a6-a109-4abd-88ef-bbbec1c775db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:53.095076 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7fb1a6-a109-4abd-88ef-bbbec1c775db admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:53.095356 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7fb1a6-a109-4abd-88ef-bbbec1c775db admin admin] PATH_INFO: `/v3/projects/a42f7b52e8e94476848366d8d3e075fc` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:53.102855 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7fb1a6-a109-4abd-88ef-bbbec1c775db admin admin] RBAC: Authorizing `identity:delete_project(project_id=a42f7b52e8e94476848366d8d3e075fc)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:53.104936 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7fb1a6-a109-4abd-88ef-bbbec1c775db admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:53.217539 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-ab7fb1a6-a109-4abd-88ef-bbbec1c775db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a42f7b52e8e94476848366d8d3e075fc', 'request_id': 'req-ab7fb1a6-a109-4abd-88ef-bbbec1c775db'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:53.233119 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-ab7fb1a6-a109-4abd-88ef-bbbec1c775db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a42f7b52e8e94476848366d8d3e075fc', 'request_id': 'req-ab7fb1a6-a109-4abd-88ef-bbbec1c775db'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:53.260346 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-ab7fb1a6-a109-4abd-88ef-bbbec1c775db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a42f7b52e8e94476848366d8d3e075fc', 'request_id': 'req-ab7fb1a6-a109-4abd-88ef-bbbec1c775db'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:53.287643 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-ab7fb1a6-a109-4abd-88ef-bbbec1c775db admin admin] The token cache is being invalidate because project a42f7b52e8e94476848366d8d3e075fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81618) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:49:53.289789 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-ab7fb1a6-a109-4abd-88ef-bbbec1c775db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:53.292316 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-ab7fb1a6-a109-4abd-88ef-bbbec1c775db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:53.306161 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1309/2632] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:49:52 2022] DELETE /identity/v3/projects/a42f7b52e8e94476848366d8d3e075fc => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 16:49:53.310204 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136ba814-6df1-4720-aaeb-c131facac8bd admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:53.312406 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-136ba814-6df1-4720-aaeb-c131facac8bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:53.334885 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136ba814-6df1-4720-aaeb-c131facac8bd admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:53.446075 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07b9265-5d47-4c83-aac3-dbe395759135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:53.448754 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07b9265-5d47-4c83-aac3-dbe395759135 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:53.449052 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07b9265-5d47-4c83-aac3-dbe395759135 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:53.449393 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07b9265-5d47-4c83-aac3-dbe395759135 admin admin] PATH_INFO: `/v3/users/3c79bcbee4d142b1a19efb5b01c2e12f` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:53.452370 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07b9265-5d47-4c83-aac3-dbe395759135 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c79bcbee4d142b1a19efb5b01c2e12f)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:53.456123 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07b9265-5d47-4c83-aac3-dbe395759135 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:53.527575 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-f07b9265-5d47-4c83-aac3-dbe395759135 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c79bcbee4d142b1a19efb5b01c2e12f'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:53.528277 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-f07b9265-5d47-4c83-aac3-dbe395759135 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c79bcbee4d142b1a19efb5b01c2e12f'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:53.538513 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-f07b9265-5d47-4c83-aac3-dbe395759135 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c79bcbee4d142b1a19efb5b01c2e12f'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:53.553397 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-f07b9265-5d47-4c83-aac3-dbe395759135 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c79bcbee4d142b1a19efb5b01c2e12f'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:53.554152 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-f07b9265-5d47-4c83-aac3-dbe395759135 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c79bcbee4d142b1a19efb5b01c2e12f'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:53.573076 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1324/2633] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:49:53 2022] DELETE /identity/v3/users/3c79bcbee4d142b1a19efb5b01c2e12f => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:49:53.946778 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7fb1a6-a109-4abd-88ef-bbbec1c775db admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:53.948211 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-ab7fb1a6-a109-4abd-88ef-bbbec1c775db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:53.978903 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7fb1a6-a109-4abd-88ef-bbbec1c775db admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:54.095764 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5661bcb-6341-4562-99ad-2a9f4ce278f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:54.096544 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5661bcb-6341-4562-99ad-2a9f4ce278f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:54.096985 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5661bcb-6341-4562-99ad-2a9f4ce278f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:54.097276 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5661bcb-6341-4562-99ad-2a9f4ce278f7 admin admin] PATH_INFO: `/v3/projects/ec3c463fd22d45e4abb9f88415db23c5` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:54.098634 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5661bcb-6341-4562-99ad-2a9f4ce278f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec3c463fd22d45e4abb9f88415db23c5)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:54.101310 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5661bcb-6341-4562-99ad-2a9f4ce278f7 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:54.160439 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07b9265-5d47-4c83-aac3-dbe395759135 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:54.161982 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-f07b9265-5d47-4c83-aac3-dbe395759135 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:54.170635 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-f5661bcb-6341-4562-99ad-2a9f4ce278f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec3c463fd22d45e4abb9f88415db23c5', 'request_id': 'req-f5661bcb-6341-4562-99ad-2a9f4ce278f7'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:54.172297 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:54.180057 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07b9265-5d47-4c83-aac3-dbe395759135 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:54.187798 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-f5661bcb-6341-4562-99ad-2a9f4ce278f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec3c463fd22d45e4abb9f88415db23c5', 'request_id': 'req-f5661bcb-6341-4562-99ad-2a9f4ce278f7'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:54.206033 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-f5661bcb-6341-4562-99ad-2a9f4ce278f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec3c463fd22d45e4abb9f88415db23c5', 'request_id': 'req-f5661bcb-6341-4562-99ad-2a9f4ce278f7'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:54.221680 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-f5661bcb-6341-4562-99ad-2a9f4ce278f7 admin admin] The token cache is being invalidate because project ec3c463fd22d45e4abb9f88415db23c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81618) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:49:54.221974 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-f5661bcb-6341-4562-99ad-2a9f4ce278f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:54.222656 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-f5661bcb-6341-4562-99ad-2a9f4ce278f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:54.225077 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1310/2634] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:49:53 2022] DELETE /identity/v3/projects/ec3c463fd22d45e4abb9f88415db23c5 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:49:54.233646 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e80db83-d4be-41fd-a0b5-eccaa99e5b20 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:54.233910 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e80db83-d4be-41fd-a0b5-eccaa99e5b20 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:54.234139 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e80db83-d4be-41fd-a0b5-eccaa99e5b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:54.242874 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:54.273544 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-6e80db83-d4be-41fd-a0b5-eccaa99e5b20 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:49:54.274833 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-6e80db83-d4be-41fd-a0b5-eccaa99e5b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:54.301305 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c5419a-4f4f-49ad-ac8f-8badaf457a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:54.302340 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c5419a-4f4f-49ad-ac8f-8badaf457a10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:54.302340 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c5419a-4f4f-49ad-ac8f-8badaf457a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:54.302559 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c5419a-4f4f-49ad-ac8f-8badaf457a10 admin admin] PATH_INFO: `/v3/users/91df23aa51134c29aa4f09640fa87b33` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:54.304414 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c5419a-4f4f-49ad-ac8f-8badaf457a10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91df23aa51134c29aa4f09640fa87b33)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:54.308477 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c5419a-4f4f-49ad-ac8f-8badaf457a10 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:54.352266 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1311/2635] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:49:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:49:54.361263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da9c410-4e96-48c1-813d-bc76b080665e None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:54.362111 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da9c410-4e96-48c1-813d-bc76b080665e None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:54.362803 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da9c410-4e96-48c1-813d-bc76b080665e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:54.399141 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-a3c5419a-4f4f-49ad-ac8f-8badaf457a10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91df23aa51134c29aa4f09640fa87b33'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:54.399593 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-1da9c410-4e96-48c1-813d-bc76b080665e None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:49:54.400698 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-1da9c410-4e96-48c1-813d-bc76b080665e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:54.400995 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-a3c5419a-4f4f-49ad-ac8f-8badaf457a10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91df23aa51134c29aa4f09640fa87b33'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:54.411193 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-a3c5419a-4f4f-49ad-ac8f-8badaf457a10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91df23aa51134c29aa4f09640fa87b33'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:54.414751 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1312/2636] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:49:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:49:54.418555 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da9c410-4e96-48c1-813d-bc76b080665e None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:54.431736 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-a3c5419a-4f4f-49ad-ac8f-8badaf457a10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91df23aa51134c29aa4f09640fa87b33'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:54.432911 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-a3c5419a-4f4f-49ad-ac8f-8badaf457a10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91df23aa51134c29aa4f09640fa87b33'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:54.433795 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da9c410-4e96-48c1-813d-bc76b080665e None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:54.442294 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b511e63-c61b-4d49-8dd8-728c12ba0da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:54.443783 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b511e63-c61b-4d49-8dd8-728c12ba0da1 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:54.444352 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b511e63-c61b-4d49-8dd8-728c12ba0da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:54.444940 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b511e63-c61b-4d49-8dd8-728c12ba0da1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:54.446600 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b511e63-c61b-4d49-8dd8-728c12ba0da1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:49:54.448829 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b511e63-c61b-4d49-8dd8-728c12ba0da1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:54.449105 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b511e63-c61b-4d49-8dd8-728c12ba0da1 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:54.464526 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1325/2637] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:49:54 2022] DELETE /identity/v3/users/91df23aa51134c29aa4f09640fa87b33 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 16:49:54.469266 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7b51d-431f-4bfc-860d-152a54e3296e None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:54.469603 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7b51d-431f-4bfc-860d-152a54e3296e None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:54.469887 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7b51d-431f-4bfc-860d-152a54e3296e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:54.476257 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1313/2638] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:49:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:49:54.484056 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b511e63-c61b-4d49-8dd8-728c12ba0da1 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:54.495462 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b511e63-c61b-4d49-8dd8-728c12ba0da1 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:54.518224 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-eab7b51d-431f-4bfc-860d-152a54e3296e None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:49:54.519543 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-eab7b51d-431f-4bfc-860d-152a54e3296e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:54.649268 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad5bbcc-7a11-46e1-82d9-67f10fd77e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:54.651474 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5bbcc-7a11-46e1-82d9-67f10fd77e6c admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:54.651887 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5bbcc-7a11-46e1-82d9-67f10fd77e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:54.654394 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5bbcc-7a11-46e1-82d9-67f10fd77e6c admin admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:54.655350 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad5bbcc-7a11-46e1-82d9-67f10fd77e6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:54.658449 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad5bbcc-7a11-46e1-82d9-67f10fd77e6c admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:54.676125 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1326/2639] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:49:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:49:54.689606 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab7b51d-431f-4bfc-860d-152a54e3296e None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:54.691805 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-eab7b51d-431f-4bfc-860d-152a54e3296e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:54.710419 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1314/2640] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:49:54 2022] POST /identity/v3/projects => generated 369 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:49:54.713578 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad5bbcc-7a11-46e1-82d9-67f10fd77e6c admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:54.810577 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab7b51d-431f-4bfc-860d-152a54e3296e None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:54.818569 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:54.832086 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:54.861415 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad5bbcc-7a11-46e1-82d9-67f10fd77e6c admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:54.871515 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7c9b76-9f2b-4339-bee5-e3b2fa0ec0f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:54.874793 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7c9b76-9f2b-4339-bee5-e3b2fa0ec0f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:54.875298 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7c9b76-9f2b-4339-bee5-e3b2fa0ec0f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:54.875747 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7c9b76-9f2b-4339-bee5-e3b2fa0ec0f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:54.877141 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7c9b76-9f2b-4339-bee5-e3b2fa0ec0f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:54.881661 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7c9b76-9f2b-4339-bee5-e3b2fa0ec0f0 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:54.930020 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1315/2641] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:49:54 2022] POST /identity/v3/users => generated 285 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:49:54.932773 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7c9b76-9f2b-4339-bee5-e3b2fa0ec0f0 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:54.947481 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7c9b76-9f2b-4339-bee5-e3b2fa0ec0f0 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:54.953352 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2acd8b-03ba-4cba-84b0-ce53df50b0ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:54.954023 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2acd8b-03ba-4cba-84b0-ce53df50b0ee admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:54.954249 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2acd8b-03ba-4cba-84b0-ce53df50b0ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:54.954719 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2acd8b-03ba-4cba-84b0-ce53df50b0ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:54.955434 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ac0aec-e770-4b3e-9012-f8914f439951 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:54.955592 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2acd8b-03ba-4cba-84b0-ce53df50b0ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:54.956112 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac0aec-e770-4b3e-9012-f8914f439951 service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:54.956389 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac0aec-e770-4b3e-9012-f8914f439951 service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:54.956672 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac0aec-e770-4b3e-9012-f8914f439951 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:54.957184 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2acd8b-03ba-4cba-84b0-ce53df50b0ee admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:54.973109 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1316/2642] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:49:54 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:49:54.974981 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2acd8b-03ba-4cba-84b0-ce53df50b0ee admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:54.984123 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ac0aec-e770-4b3e-9012-f8914f439951 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:54.987067 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ac0aec-e770-4b3e-9012-f8914f439951 service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:54.989120 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2acd8b-03ba-4cba-84b0-ce53df50b0ee admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:54.995055 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e570b7-8582-4508-9fc4-253913f53770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:54.995754 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e570b7-8582-4508-9fc4-253913f53770 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:54.996083 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e570b7-8582-4508-9fc4-253913f53770 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:54.996359 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e570b7-8582-4508-9fc4-253913f53770 admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:54.997136 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e570b7-8582-4508-9fc4-253913f53770 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:54.998925 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e570b7-8582-4508-9fc4-253913f53770 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:55.010724 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1327/2643] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:49:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 16:49:55.016986 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1317/2644] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:49:54 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 16:49:55.018306 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ac0aec-e770-4b3e-9012-f8914f439951 service neutron] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:55.034176 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ac0aec-e770-4b3e-9012-f8914f439951 service neutron] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:55.042121 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c189bd9-4cdd-4426-b9f3-97662edcffcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:55.042478 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c189bd9-4cdd-4426-b9f3-97662edcffcb admin admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:55.042868 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c189bd9-4cdd-4426-b9f3-97662edcffcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:55.043248 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c189bd9-4cdd-4426-b9f3-97662edcffcb admin admin] PATH_INFO: `/v3/projects/dd0170949e9d4e5cb1ce092558d970ee/users/9164b586372c4a9c82463c860501012e/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:55.062642 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c189bd9-4cdd-4426-b9f3-97662edcffcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd0170949e9d4e5cb1ce092558d970ee, user_id=9164b586372c4a9c82463c860501012e, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:55.065482 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c189bd9-4cdd-4426-b9f3-97662edcffcb admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:55.098059 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1328/2645] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:49:55 2022] PUT /identity/v3/projects/dd0170949e9d4e5cb1ce092558d970ee/users/9164b586372c4a9c82463c860501012e/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:49:55.105666 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05746862-5161-4bfb-aaa2-57f5573d4ac5 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:55.107050 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05746862-5161-4bfb-aaa2-57f5573d4ac5 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:55.108000 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05746862-5161-4bfb-aaa2-57f5573d4ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:55.166546 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-05746862-5161-4bfb-aaa2-57f5573d4ac5 None None] MFA Rules not processed for user `9164b586372c4a9c82463c860501012e`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:49:55.175999 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-05746862-5161-4bfb-aaa2-57f5573d4ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:55.289918 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1318/2646] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:49:55 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:49:55.303316 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c189bd9-4cdd-4426-b9f3-97662edcffcb admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:55.305197 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-3c189bd9-4cdd-4426-b9f3-97662edcffcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:55.385210 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c189bd9-4cdd-4426-b9f3-97662edcffcb admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:55.389812 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d295a380-1640-4177-95e0-35b71be75a45 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5632ade0cc6f4ae28f72160de68486cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:55.390555 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d295a380-1640-4177-95e0-35b71be75a45 service glance] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:55.391382 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d295a380-1640-4177-95e0-35b71be75a45 service glance] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:55.391742 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d295a380-1640-4177-95e0-35b71be75a45 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:55.403967 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d295a380-1640-4177-95e0-35b71be75a45 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:55.405409 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d295a380-1640-4177-95e0-35b71be75a45 service glance] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:55.418275 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1329/2647] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:49:55 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:49:55.476245 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05746862-5161-4bfb-aaa2-57f5573d4ac5 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:55.478399 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-05746862-5161-4bfb-aaa2-57f5573d4ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:55.597291 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05746862-5161-4bfb-aaa2-57f5573d4ac5 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:55.602332 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3524f1-bd20-467d-9469-0dc326691995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:55.603120 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3524f1-bd20-467d-9469-0dc326691995 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:55.603439 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3524f1-bd20-467d-9469-0dc326691995 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:55.603710 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3524f1-bd20-467d-9469-0dc326691995 admin admin] PATH_INFO: `/v3/projects/223a5e788741496c9e98a91cde6aadbc` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:55.604962 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3524f1-bd20-467d-9469-0dc326691995 admin admin] RBAC: Authorizing `identity:delete_project(project_id=223a5e788741496c9e98a91cde6aadbc)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:55.607283 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3524f1-bd20-467d-9469-0dc326691995 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:55.671494 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:55.676990 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-4e3524f1-bd20-467d-9469-0dc326691995 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '223a5e788741496c9e98a91cde6aadbc', 'request_id': 'req-4e3524f1-bd20-467d-9469-0dc326691995'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:55.686294 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-4e3524f1-bd20-467d-9469-0dc326691995 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '223a5e788741496c9e98a91cde6aadbc', 'request_id': 'req-4e3524f1-bd20-467d-9469-0dc326691995'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:55.695938 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-4e3524f1-bd20-467d-9469-0dc326691995 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '223a5e788741496c9e98a91cde6aadbc', 'request_id': 'req-4e3524f1-bd20-467d-9469-0dc326691995'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:55.717904 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-4e3524f1-bd20-467d-9469-0dc326691995 admin admin] The token cache is being invalidate because project 223a5e788741496c9e98a91cde6aadbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81618) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:49:55.718494 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-4e3524f1-bd20-467d-9469-0dc326691995 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:55.719225 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-4e3524f1-bd20-467d-9469-0dc326691995 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:55.721845 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1319/2648] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:49:55 2022] DELETE /identity/v3/projects/223a5e788741496c9e98a91cde6aadbc => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:49:55.725676 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d295a380-1640-4177-95e0-35b71be75a45 service glance] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:55.738813 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d295a380-1640-4177-95e0-35b71be75a45 service glance] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:55.848104 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b84e18-816f-407a-9c8c-1ed4c6846774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:55.848857 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b84e18-816f-407a-9c8c-1ed4c6846774 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:55.849201 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b84e18-816f-407a-9c8c-1ed4c6846774 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:55.849548 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b84e18-816f-407a-9c8c-1ed4c6846774 admin admin] PATH_INFO: `/v3/users/765ec0dfea0441dd952ea8a1ef693a5b` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:55.851297 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b84e18-816f-407a-9c8c-1ed4c6846774 admin admin] RBAC: Authorizing `identity:delete_user(user_id=765ec0dfea0441dd952ea8a1ef693a5b)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:55.857919 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b84e18-816f-407a-9c8c-1ed4c6846774 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:55.941632 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:55.956239 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-73b84e18-816f-407a-9c8c-1ed4c6846774 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '765ec0dfea0441dd952ea8a1ef693a5b'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:55.958244 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-73b84e18-816f-407a-9c8c-1ed4c6846774 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '765ec0dfea0441dd952ea8a1ef693a5b'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:55.973091 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-73b84e18-816f-407a-9c8c-1ed4c6846774 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '765ec0dfea0441dd952ea8a1ef693a5b'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:56.007557 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-73b84e18-816f-407a-9c8c-1ed4c6846774 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '765ec0dfea0441dd952ea8a1ef693a5b'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:56.013658 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-73b84e18-816f-407a-9c8c-1ed4c6846774 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '765ec0dfea0441dd952ea8a1ef693a5b'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:56.059351 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1330/2649] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:49:55 2022] DELETE /identity/v3/users/765ec0dfea0441dd952ea8a1ef693a5b => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:49:56.522135 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3524f1-bd20-467d-9469-0dc326691995 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:56.524620 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-4e3524f1-bd20-467d-9469-0dc326691995 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:56.553738 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3524f1-bd20-467d-9469-0dc326691995 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:56.639746 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeac3f82-12c1-45cb-8a7a-5113956e5960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:56.640395 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac3f82-12c1-45cb-8a7a-5113956e5960 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:56.640717 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac3f82-12c1-45cb-8a7a-5113956e5960 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:56.640946 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac3f82-12c1-45cb-8a7a-5113956e5960 admin admin] PATH_INFO: `/v3/projects/21d5dc352dfa45b7865125b8137fae28` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:56.642176 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeac3f82-12c1-45cb-8a7a-5113956e5960 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21d5dc352dfa45b7865125b8137fae28)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:56.644158 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeac3f82-12c1-45cb-8a7a-5113956e5960 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:56.714397 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-eeac3f82-12c1-45cb-8a7a-5113956e5960 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21d5dc352dfa45b7865125b8137fae28', 'request_id': 'req-eeac3f82-12c1-45cb-8a7a-5113956e5960'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:56.722505 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-eeac3f82-12c1-45cb-8a7a-5113956e5960 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21d5dc352dfa45b7865125b8137fae28', 'request_id': 'req-eeac3f82-12c1-45cb-8a7a-5113956e5960'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:56.739451 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-eeac3f82-12c1-45cb-8a7a-5113956e5960 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21d5dc352dfa45b7865125b8137fae28', 'request_id': 'req-eeac3f82-12c1-45cb-8a7a-5113956e5960'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:56.761094 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-eeac3f82-12c1-45cb-8a7a-5113956e5960 admin admin] The token cache is being invalidate because project 21d5dc352dfa45b7865125b8137fae28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81618) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:49:56.761376 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-eeac3f82-12c1-45cb-8a7a-5113956e5960 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:56.762050 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-eeac3f82-12c1-45cb-8a7a-5113956e5960 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:56.764073 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1320/2650] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:49:56 2022] DELETE /identity/v3/projects/21d5dc352dfa45b7865125b8137fae28 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 16:49:56.769135 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6490d96c-1709-4fc0-82a6-28155b4b733d None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:56.769692 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6490d96c-1709-4fc0-82a6-28155b4b733d None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:56.769842 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6490d96c-1709-4fc0-82a6-28155b4b733d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:56.796305 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-6490d96c-1709-4fc0-82a6-28155b4b733d None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:49:56.797879 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-6490d96c-1709-4fc0-82a6-28155b4b733d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:56.902828 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1331/2651] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:49:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:49:56.911258 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7893a5-932c-4bbc-98d5-98baf6f6cdf0 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:56.911590 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7893a5-932c-4bbc-98d5-98baf6f6cdf0 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:56.911902 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7893a5-932c-4bbc-98d5-98baf6f6cdf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:56.933213 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-1e7893a5-932c-4bbc-98d5-98baf6f6cdf0 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:49:56.934471 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-1e7893a5-932c-4bbc-98d5-98baf6f6cdf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:56.942529 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1321/2652] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:49:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:49:56.947656 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6490d96c-1709-4fc0-82a6-28155b4b733d None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:56.959839 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6490d96c-1709-4fc0-82a6-28155b4b733d None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:56.973711 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffea731-f71f-43d1-aab0-30d7191424d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:56.974530 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffea731-f71f-43d1-aab0-30d7191424d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:56.974814 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffea731-f71f-43d1-aab0-30d7191424d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:56.975088 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffea731-f71f-43d1-aab0-30d7191424d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:56.975920 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffea731-f71f-43d1-aab0-30d7191424d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:49:56.976283 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffea731-f71f-43d1-aab0-30d7191424d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:56.978710 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffea731-f71f-43d1-aab0-30d7191424d6 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:56.993275 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1332/2653] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:49:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:49:56.998356 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7893a5-932c-4bbc-98d5-98baf6f6cdf0 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:57.009264 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7893a5-932c-4bbc-98d5-98baf6f6cdf0 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:57.014739 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24180c41-63a9-4a88-bf11-156dbc8b0a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:57.015490 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24180c41-63a9-4a88-bf11-156dbc8b0a13 admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:57.015829 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24180c41-63a9-4a88-bf11-156dbc8b0a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:57.016063 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24180c41-63a9-4a88-bf11-156dbc8b0a13 admin admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:57.016789 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24180c41-63a9-4a88-bf11-156dbc8b0a13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:57.018814 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24180c41-63a9-4a88-bf11-156dbc8b0a13 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:57.054342 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1322/2654] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:49:56 2022] POST /identity/v3/projects => generated 379 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:49:57.059984 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffea731-f71f-43d1-aab0-30d7191424d6 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:57.072463 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffea731-f71f-43d1-aab0-30d7191424d6 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:57.244792 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfec725b-8cc0-4a3d-8875-bb7880853ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:57.245078 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec725b-8cc0-4a3d-8875-bb7880853ba5 admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:57.245202 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec725b-8cc0-4a3d-8875-bb7880853ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:57.245407 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec725b-8cc0-4a3d-8875-bb7880853ba5 admin admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:57.246050 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfec725b-8cc0-4a3d-8875-bb7880853ba5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:57.252000 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfec725b-8cc0-4a3d-8875-bb7880853ba5 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:57.305252 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1333/2655] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:49:57 2022] POST /identity/v3/users => generated 290 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 16:49:57.309564 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24180c41-63a9-4a88-bf11-156dbc8b0a13 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:57.339318 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24180c41-63a9-4a88-bf11-156dbc8b0a13 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:57.348555 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0655ca-0894-4743-b540-746d2cb84fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:57.349647 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0655ca-0894-4743-b540-746d2cb84fce admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:57.350151 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0655ca-0894-4743-b540-746d2cb84fce admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:57.350701 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0655ca-0894-4743-b540-746d2cb84fce admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:57.352449 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0655ca-0894-4743-b540-746d2cb84fce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:57.356260 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0655ca-0894-4743-b540-746d2cb84fce admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:57.400140 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1323/2656] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:49:57 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:49:57.404130 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfec725b-8cc0-4a3d-8875-bb7880853ba5 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:57.430944 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfec725b-8cc0-4a3d-8875-bb7880853ba5 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:57.430944 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe675b11-4821-4ac1-a8c5-f98e646512f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:57.430944 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe675b11-4821-4ac1-a8c5-f98e646512f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:57.430944 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe675b11-4821-4ac1-a8c5-f98e646512f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:57.431770 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe675b11-4821-4ac1-a8c5-f98e646512f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:57.431770 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe675b11-4821-4ac1-a8c5-f98e646512f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:57.431770 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe675b11-4821-4ac1-a8c5-f98e646512f5 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:57.458424 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1334/2657] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:49:57 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:49:57.460469 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0655ca-0894-4743-b540-746d2cb84fce admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:57.473419 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0655ca-0894-4743-b540-746d2cb84fce admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:57.479226 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ba792e-557a-4ff9-94e4-961784ba1e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:57.480070 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ba792e-557a-4ff9-94e4-961784ba1e1d admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:57.480455 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ba792e-557a-4ff9-94e4-961784ba1e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:57.480786 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ba792e-557a-4ff9-94e4-961784ba1e1d admin admin] PATH_INFO: `/v3/projects/8a0fa543ab52492b8c9e35c389986f43/users/6161d54565b84cdba50c4121e68e4f0a/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:57.498901 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ba792e-557a-4ff9-94e4-961784ba1e1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a0fa543ab52492b8c9e35c389986f43, user_id=6161d54565b84cdba50c4121e68e4f0a, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:57.501815 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ba792e-557a-4ff9-94e4-961784ba1e1d admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:57.527012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:57.542713 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1324/2658] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:49:57 2022] PUT /identity/v3/projects/8a0fa543ab52492b8c9e35c389986f43/users/6161d54565b84cdba50c4121e68e4f0a/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:49:57.549969 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b9a917-cccf-4acd-9bac-81da01a4047a None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:57.550400 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b9a917-cccf-4acd-9bac-81da01a4047a None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:57.551646 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b9a917-cccf-4acd-9bac-81da01a4047a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:57.589680 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-b5b9a917-cccf-4acd-9bac-81da01a4047a None None] MFA Rules not processed for user `6161d54565b84cdba50c4121e68e4f0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:49:57.590932 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-b5b9a917-cccf-4acd-9bac-81da01a4047a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:57.599975 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:57.680076 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1335/2659] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:49:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:49:57.685586 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ba792e-557a-4ff9-94e4-961784ba1e1d admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:57.772036 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ba792e-557a-4ff9-94e4-961784ba1e1d admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:57.779408 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d78c1a-fbb7-4cea-ba80-ea0847ed97d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:57.780154 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d78c1a-fbb7-4cea-ba80-ea0847ed97d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:57.780439 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d78c1a-fbb7-4cea-ba80-ea0847ed97d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:57.780716 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d78c1a-fbb7-4cea-ba80-ea0847ed97d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:57.781392 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d78c1a-fbb7-4cea-ba80-ea0847ed97d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:57.783479 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d78c1a-fbb7-4cea-ba80-ea0847ed97d8 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:57.817034 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1325/2660] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:49:57 2022] POST /identity/v3/projects => generated 381 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:49:57.821179 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b9a917-cccf-4acd-9bac-81da01a4047a None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:57.835730 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b9a917-cccf-4acd-9bac-81da01a4047a None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:57.904764 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065278a4-d891-4d09-a205-eb6f596b260c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:57.905496 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065278a4-d891-4d09-a205-eb6f596b260c admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:57.905837 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065278a4-d891-4d09-a205-eb6f596b260c admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:57.906173 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065278a4-d891-4d09-a205-eb6f596b260c admin admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:57.906916 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065278a4-d891-4d09-a205-eb6f596b260c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:57.908912 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065278a4-d891-4d09-a205-eb6f596b260c admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:57.942125 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1336/2661] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:49:57 2022] POST /identity/v3/users => generated 297 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:49:57.943241 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d78c1a-fbb7-4cea-ba80-ea0847ed97d8 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:57.955195 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d78c1a-fbb7-4cea-ba80-ea0847ed97d8 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:57.960543 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943851ce-c33d-413c-ad02-e02062753a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:57.962216 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943851ce-c33d-413c-ad02-e02062753a8d admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:57.962829 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943851ce-c33d-413c-ad02-e02062753a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:57.963232 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943851ce-c33d-413c-ad02-e02062753a8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:57.963923 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943851ce-c33d-413c-ad02-e02062753a8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:57.965596 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943851ce-c33d-413c-ad02-e02062753a8d admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:57.978352 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1326/2662] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:49:57 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:49:57.982719 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065278a4-d891-4d09-a205-eb6f596b260c admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:57.993624 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065278a4-d891-4d09-a205-eb6f596b260c admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:57.998687 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128624d2-361a-404e-8604-d048a84e1007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:57.999356 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128624d2-361a-404e-8604-d048a84e1007 admin admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:57.999649 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128624d2-361a-404e-8604-d048a84e1007 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:57.999921 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128624d2-361a-404e-8604-d048a84e1007 admin admin] PATH_INFO: `/v3/projects/376149162f024f48b244ce47e69f3bc6/users/c0a831bcdafd42cebe8160885ca6072c/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:58.017000 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128624d2-361a-404e-8604-d048a84e1007 admin admin] RBAC: Authorizing `identity:create_grant(project_id=376149162f024f48b244ce47e69f3bc6, user_id=c0a831bcdafd42cebe8160885ca6072c, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:58.019769 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128624d2-361a-404e-8604-d048a84e1007 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:58.037508 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1337/2663] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:49:57 2022] PUT /identity/v3/projects/376149162f024f48b244ce47e69f3bc6/users/c0a831bcdafd42cebe8160885ca6072c/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:49:58.042792 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943851ce-c33d-413c-ad02-e02062753a8d admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:58.060050 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943851ce-c33d-413c-ad02-e02062753a8d admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:58.140552 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5511fdaf-740d-45db-b894-59d0ecf6c3aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:58.141274 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511fdaf-740d-45db-b894-59d0ecf6c3aa admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:58.141627 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511fdaf-740d-45db-b894-59d0ecf6c3aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:58.141984 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511fdaf-740d-45db-b894-59d0ecf6c3aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:58.142948 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5511fdaf-740d-45db-b894-59d0ecf6c3aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:58.144747 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5511fdaf-740d-45db-b894-59d0ecf6c3aa admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:58.156363 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1327/2664] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:49:58 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:49:58.164946 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128624d2-361a-404e-8604-d048a84e1007 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:58.183289 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128624d2-361a-404e-8604-d048a84e1007 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:58.189006 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09631b7e-da5d-43db-b393-da4834e24499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:58.190028 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09631b7e-da5d-43db-b393-da4834e24499 admin admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:58.190595 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09631b7e-da5d-43db-b393-da4834e24499 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:58.191028 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09631b7e-da5d-43db-b393-da4834e24499 admin admin] PATH_INFO: `/v3/projects/376149162f024f48b244ce47e69f3bc6/users/c0a831bcdafd42cebe8160885ca6072c/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:58.193609 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09631b7e-da5d-43db-b393-da4834e24499 admin admin] RBAC: Authorizing `identity:create_grant(project_id=376149162f024f48b244ce47e69f3bc6, user_id=c0a831bcdafd42cebe8160885ca6072c, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:58.198034 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09631b7e-da5d-43db-b393-da4834e24499 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:58.211793 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1338/2665] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:49:58 2022] PUT /identity/v3/projects/376149162f024f48b244ce47e69f3bc6/users/c0a831bcdafd42cebe8160885ca6072c/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:49:58.231524 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d134889-ba3e-4423-831e-2937d6669063 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:58.231524 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d134889-ba3e-4423-831e-2937d6669063 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:58.231524 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d134889-ba3e-4423-831e-2937d6669063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:58.263568 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-3d134889-ba3e-4423-831e-2937d6669063 None None] MFA Rules not processed for user `c0a831bcdafd42cebe8160885ca6072c`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:49:58.264827 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-3d134889-ba3e-4423-831e-2937d6669063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:58.348263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:58.360241 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1328/2666] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:49:58 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:49:58.370784 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09631b7e-da5d-43db-b393-da4834e24499 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:58.372949 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-09631b7e-da5d-43db-b393-da4834e24499 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:58.444617 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09631b7e-da5d-43db-b393-da4834e24499 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:58.450533 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6069fa-882f-46d2-be69-bc47cd05abab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:58.450533 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6069fa-882f-46d2-be69-bc47cd05abab service cinder] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:58.450533 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6069fa-882f-46d2-be69-bc47cd05abab service cinder] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:58.450533 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6069fa-882f-46d2-be69-bc47cd05abab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:58.460764 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6069fa-882f-46d2-be69-bc47cd05abab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:58.462224 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6069fa-882f-46d2-be69-bc47cd05abab service cinder] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:58.474210 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1339/2667] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:49:58 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:49:59.018544 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d134889-ba3e-4423-831e-2937d6669063 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:59.113103 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d134889-ba3e-4423-831e-2937d6669063 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:59.118590 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2509f9e1-741d-4bca-bceb-eba226547a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:59.119282 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2509f9e1-741d-4bca-bceb-eba226547a34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:59.119600 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2509f9e1-741d-4bca-bceb-eba226547a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:59.119839 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2509f9e1-741d-4bca-bceb-eba226547a34 admin admin] PATH_INFO: `/v3/users/6161d54565b84cdba50c4121e68e4f0a` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:59.121132 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2509f9e1-741d-4bca-bceb-eba226547a34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6161d54565b84cdba50c4121e68e4f0a)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:59.123103 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2509f9e1-741d-4bca-bceb-eba226547a34 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:59.184249 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-2509f9e1-741d-4bca-bceb-eba226547a34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6161d54565b84cdba50c4121e68e4f0a'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:59.192099 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-2509f9e1-741d-4bca-bceb-eba226547a34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6161d54565b84cdba50c4121e68e4f0a'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:59.206831 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-2509f9e1-741d-4bca-bceb-eba226547a34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6161d54565b84cdba50c4121e68e4f0a'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:59.215643 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-2509f9e1-741d-4bca-bceb-eba226547a34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6161d54565b84cdba50c4121e68e4f0a'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:59.216290 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-2509f9e1-741d-4bca-bceb-eba226547a34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6161d54565b84cdba50c4121e68e4f0a'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:59.222443 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1329/2668] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:49:59 2022] DELETE /identity/v3/users/6161d54565b84cdba50c4121e68e4f0a => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:49:59.227673 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c3e0bb-5fae-44d4-9e4f-0b396b3e3250 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:59.228154 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c3e0bb-5fae-44d4-9e4f-0b396b3e3250 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:59.228288 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c3e0bb-5fae-44d4-9e4f-0b396b3e3250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:59.250824 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:59.272214 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-82c3e0bb-5fae-44d4-9e4f-0b396b3e3250 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:49:59.273610 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-82c3e0bb-5fae-44d4-9e4f-0b396b3e3250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:59.352525 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1340/2669] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:49:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:49:59.362877 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2509f9e1-741d-4bca-bceb-eba226547a34 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:59.364293 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-2509f9e1-741d-4bca-bceb-eba226547a34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:59.450809 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2509f9e1-741d-4bca-bceb-eba226547a34 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:59.450809 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae24a7a2-5985-4ac1-a965-828580aceef2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:59.450809 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae24a7a2-5985-4ac1-a965-828580aceef2 service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:59.450809 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae24a7a2-5985-4ac1-a965-828580aceef2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:59.451433 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae24a7a2-5985-4ac1-a965-828580aceef2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:59.469509 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae24a7a2-5985-4ac1-a965-828580aceef2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:59.471111 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae24a7a2-5985-4ac1-a965-828580aceef2 service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:59.486158 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1330/2670] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:49:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:49:59.846560 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c3e0bb-5fae-44d4-9e4f-0b396b3e3250 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:49:59.849141 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-82c3e0bb-5fae-44d4-9e4f-0b396b3e3250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:49:59.891941 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c3e0bb-5fae-44d4-9e4f-0b396b3e3250 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:49:59.896518 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35882eb7-990f-4fc4-a9a5-491438fc86c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:49:59.897160 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35882eb7-990f-4fc4-a9a5-491438fc86c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:49:59.897431 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35882eb7-990f-4fc4-a9a5-491438fc86c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:49:59.897700 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35882eb7-990f-4fc4-a9a5-491438fc86c3 admin admin] PATH_INFO: `/v3/projects/8a0fa543ab52492b8c9e35c389986f43` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:49:59.898889 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35882eb7-990f-4fc4-a9a5-491438fc86c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a0fa543ab52492b8c9e35c389986f43)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:49:59.900785 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35882eb7-990f-4fc4-a9a5-491438fc86c3 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:49:59.952210 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-35882eb7-990f-4fc4-a9a5-491438fc86c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a0fa543ab52492b8c9e35c389986f43', 'request_id': 'req-35882eb7-990f-4fc4-a9a5-491438fc86c3'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:59.959377 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-35882eb7-990f-4fc4-a9a5-491438fc86c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a0fa543ab52492b8c9e35c389986f43', 'request_id': 'req-35882eb7-990f-4fc4-a9a5-491438fc86c3'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:59.966161 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-35882eb7-990f-4fc4-a9a5-491438fc86c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a0fa543ab52492b8c9e35c389986f43', 'request_id': 'req-35882eb7-990f-4fc4-a9a5-491438fc86c3'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:59.983655 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-35882eb7-990f-4fc4-a9a5-491438fc86c3 admin admin] The token cache is being invalidate because project 8a0fa543ab52492b8c9e35c389986f43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81617) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:49:59.984040 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-35882eb7-990f-4fc4-a9a5-491438fc86c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:59.984821 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-35882eb7-990f-4fc4-a9a5-491438fc86c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:49:59.986642 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1341/2671] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:49:59 2022] DELETE /identity/v3/projects/8a0fa543ab52492b8c9e35c389986f43 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:49:59.990422 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae24a7a2-5985-4ac1-a965-828580aceef2 service neutron] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:00.001715 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae24a7a2-5985-4ac1-a965-828580aceef2 service neutron] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:00.014993 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:50:00.088119 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa2c5b3-35e2-4874-8bc1-65c42b54bc8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:00.088842 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa2c5b3-35e2-4874-8bc1-65c42b54bc8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:00.089040 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa2c5b3-35e2-4874-8bc1-65c42b54bc8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:00.089324 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa2c5b3-35e2-4874-8bc1-65c42b54bc8e admin admin] PATH_INFO: `/v3/users/c0a831bcdafd42cebe8160885ca6072c` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:00.090544 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa2c5b3-35e2-4874-8bc1-65c42b54bc8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0a831bcdafd42cebe8160885ca6072c)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:00.092463 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa2c5b3-35e2-4874-8bc1-65c42b54bc8e admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:00.149953 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-eaa2c5b3-35e2-4874-8bc1-65c42b54bc8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0a831bcdafd42cebe8160885ca6072c'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:00.157800 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-eaa2c5b3-35e2-4874-8bc1-65c42b54bc8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0a831bcdafd42cebe8160885ca6072c'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:00.170703 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-eaa2c5b3-35e2-4874-8bc1-65c42b54bc8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0a831bcdafd42cebe8160885ca6072c'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:00.180166 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-eaa2c5b3-35e2-4874-8bc1-65c42b54bc8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0a831bcdafd42cebe8160885ca6072c'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:00.180886 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-eaa2c5b3-35e2-4874-8bc1-65c42b54bc8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0a831bcdafd42cebe8160885ca6072c'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:00.188399 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1331/2672] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:49:59 2022] DELETE /identity/v3/users/c0a831bcdafd42cebe8160885ca6072c => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:50:00.517593 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35882eb7-990f-4fc4-a9a5-491438fc86c3 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:00.519064 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-35882eb7-990f-4fc4-a9a5-491438fc86c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:00.544790 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35882eb7-990f-4fc4-a9a5-491438fc86c3 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:00.625123 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf3bb9c-b4cc-4fab-ba26-08a016abbdf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:00.626036 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf3bb9c-b4cc-4fab-ba26-08a016abbdf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:00.626460 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf3bb9c-b4cc-4fab-ba26-08a016abbdf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:00.626842 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf3bb9c-b4cc-4fab-ba26-08a016abbdf4 admin admin] PATH_INFO: `/v3/projects/376149162f024f48b244ce47e69f3bc6` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:00.628194 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf3bb9c-b4cc-4fab-ba26-08a016abbdf4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=376149162f024f48b244ce47e69f3bc6)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:00.630448 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf3bb9c-b4cc-4fab-ba26-08a016abbdf4 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:00.670178 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:50:00.685866 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-4cf3bb9c-b4cc-4fab-ba26-08a016abbdf4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '376149162f024f48b244ce47e69f3bc6', 'request_id': 'req-4cf3bb9c-b4cc-4fab-ba26-08a016abbdf4'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:00.693385 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-4cf3bb9c-b4cc-4fab-ba26-08a016abbdf4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '376149162f024f48b244ce47e69f3bc6', 'request_id': 'req-4cf3bb9c-b4cc-4fab-ba26-08a016abbdf4'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:00.699939 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-4cf3bb9c-b4cc-4fab-ba26-08a016abbdf4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '376149162f024f48b244ce47e69f3bc6', 'request_id': 'req-4cf3bb9c-b4cc-4fab-ba26-08a016abbdf4'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:00.715274 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-4cf3bb9c-b4cc-4fab-ba26-08a016abbdf4 admin admin] The token cache is being invalidate because project 376149162f024f48b244ce47e69f3bc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81617) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:50:00.715655 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-4cf3bb9c-b4cc-4fab-ba26-08a016abbdf4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:00.716221 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-4cf3bb9c-b4cc-4fab-ba26-08a016abbdf4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:00.718078 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1342/2673] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:50:00 2022] DELETE /identity/v3/projects/376149162f024f48b244ce47e69f3bc6 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:50:00.727407 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e25279b-1488-4ce1-8092-c532d85753f0 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:00.727407 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e25279b-1488-4ce1-8092-c532d85753f0 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:00.727407 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e25279b-1488-4ce1-8092-c532d85753f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:00.758796 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-6e25279b-1488-4ce1-8092-c532d85753f0 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:50:00.760087 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-6e25279b-1488-4ce1-8092-c532d85753f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:00.847612 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1332/2674] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:50:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:50:00.857401 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e1e81-ecc8-42ae-82d0-ffe88c9f16e1 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:00.857661 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e1e81-ecc8-42ae-82d0-ffe88c9f16e1 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:00.857996 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e1e81-ecc8-42ae-82d0-ffe88c9f16e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:00.879643 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-5b9e1e81-ecc8-42ae-82d0-ffe88c9f16e1 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:50:00.880996 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-5b9e1e81-ecc8-42ae-82d0-ffe88c9f16e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:00.889190 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1343/2675] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:50:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:50:00.893667 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e25279b-1488-4ce1-8092-c532d85753f0 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:00.903489 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e25279b-1488-4ce1-8092-c532d85753f0 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:00.908521 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298debeb-7675-403e-a384-5c0c2533f890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:00.909234 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298debeb-7675-403e-a384-5c0c2533f890 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:00.909599 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298debeb-7675-403e-a384-5c0c2533f890 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:00.909897 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298debeb-7675-403e-a384-5c0c2533f890 admin admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:00.910675 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298debeb-7675-403e-a384-5c0c2533f890 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:50:00.911058 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298debeb-7675-403e-a384-5c0c2533f890 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:00.912669 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298debeb-7675-403e-a384-5c0c2533f890 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:00.930204 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1333/2676] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:50:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:50:00.931851 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9e1e81-ecc8-42ae-82d0-ffe88c9f16e1 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:00.942972 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9e1e81-ecc8-42ae-82d0-ffe88c9f16e1 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:00.956578 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b73d22-0cd0-41f0-bfd5-14d1792360b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:00.957390 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b73d22-0cd0-41f0-bfd5-14d1792360b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:00.957748 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b73d22-0cd0-41f0-bfd5-14d1792360b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:00.958175 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b73d22-0cd0-41f0-bfd5-14d1792360b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:00.959005 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b73d22-0cd0-41f0-bfd5-14d1792360b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:00.961778 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b73d22-0cd0-41f0-bfd5-14d1792360b1 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:00.999829 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1344/2677] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:50:00 2022] POST /identity/v3/projects => generated 383 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:50:01.001089 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298debeb-7675-403e-a384-5c0c2533f890 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:01.014388 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298debeb-7675-403e-a384-5c0c2533f890 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:01.054860 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:50:01.120801 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9409c1cc-dacf-4b40-946e-e09c6a2b18bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:01.121583 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9409c1cc-dacf-4b40-946e-e09c6a2b18bd admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:01.121955 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9409c1cc-dacf-4b40-946e-e09c6a2b18bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:01.122331 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9409c1cc-dacf-4b40-946e-e09c6a2b18bd admin admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:01.123167 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9409c1cc-dacf-4b40-946e-e09c6a2b18bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:01.125198 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9409c1cc-dacf-4b40-946e-e09c6a2b18bd admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:01.155175 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1334/2678] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:50:01 2022] POST /identity/v3/users => generated 292 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:50:01.162491 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b73d22-0cd0-41f0-bfd5-14d1792360b1 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:01.179798 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b73d22-0cd0-41f0-bfd5-14d1792360b1 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:01.185096 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7a9ad2-9354-4e91-b046-b8fd02051f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:01.185872 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a9ad2-9354-4e91-b046-b8fd02051f26 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:01.186220 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a9ad2-9354-4e91-b046-b8fd02051f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:01.186590 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a9ad2-9354-4e91-b046-b8fd02051f26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:01.187449 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7a9ad2-9354-4e91-b046-b8fd02051f26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:01.189231 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7a9ad2-9354-4e91-b046-b8fd02051f26 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:01.200041 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1345/2679] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:50:01 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 16:50:01.204088 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9409c1cc-dacf-4b40-946e-e09c6a2b18bd admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:01.215220 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9409c1cc-dacf-4b40-946e-e09c6a2b18bd admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:01.219922 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd09f8c-4ea8-4daf-a529-1d9bac97e456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:01.220522 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd09f8c-4ea8-4daf-a529-1d9bac97e456 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:01.220804 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd09f8c-4ea8-4daf-a529-1d9bac97e456 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:01.221080 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd09f8c-4ea8-4daf-a529-1d9bac97e456 admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:01.221794 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd09f8c-4ea8-4daf-a529-1d9bac97e456 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:01.223606 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd09f8c-4ea8-4daf-a529-1d9bac97e456 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:01.235958 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1335/2680] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:50:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:50:01.240282 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7a9ad2-9354-4e91-b046-b8fd02051f26 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:01.252439 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7a9ad2-9354-4e91-b046-b8fd02051f26 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:01.257472 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828823e7-30ed-469a-89c4-2c9d46ffcd23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:01.258139 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828823e7-30ed-469a-89c4-2c9d46ffcd23 admin admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:01.258454 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828823e7-30ed-469a-89c4-2c9d46ffcd23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:01.258732 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828823e7-30ed-469a-89c4-2c9d46ffcd23 admin admin] PATH_INFO: `/v3/projects/98d16937467b4c98881dc45e6025e346/users/5bfe97a0c772463baa010e62c4305793/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:01.274415 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828823e7-30ed-469a-89c4-2c9d46ffcd23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98d16937467b4c98881dc45e6025e346, user_id=5bfe97a0c772463baa010e62c4305793, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:01.277040 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828823e7-30ed-469a-89c4-2c9d46ffcd23 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:01.292652 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1346/2681] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:50:01 2022] PUT /identity/v3/projects/98d16937467b4c98881dc45e6025e346/users/5bfe97a0c772463baa010e62c4305793/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:50:01.363730 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1214cc0e-47bc-4dd0-8139-0f086c9dba9c None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:01.364014 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1214cc0e-47bc-4dd0-8139-0f086c9dba9c None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:01.364289 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1214cc0e-47bc-4dd0-8139-0f086c9dba9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:01.400042 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-1214cc0e-47bc-4dd0-8139-0f086c9dba9c None None] MFA Rules not processed for user `5bfe97a0c772463baa010e62c4305793`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:50:01.401615 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-1214cc0e-47bc-4dd0-8139-0f086c9dba9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:01.485765 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1336/2682] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:50:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:50:01.497196 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828823e7-30ed-469a-89c4-2c9d46ffcd23 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:01.498975 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-828823e7-30ed-469a-89c4-2c9d46ffcd23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:01.572460 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828823e7-30ed-469a-89c4-2c9d46ffcd23 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:01.576361 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe45a14-e4d5-43b7-89d9-dda84554f06a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:01.576980 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe45a14-e4d5-43b7-89d9-dda84554f06a service cinder] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:01.577267 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe45a14-e4d5-43b7-89d9-dda84554f06a service cinder] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:01.577697 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe45a14-e4d5-43b7-89d9-dda84554f06a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:01.587093 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe45a14-e4d5-43b7-89d9-dda84554f06a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:01.588523 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe45a14-e4d5-43b7-89d9-dda84554f06a service cinder] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:01.604101 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1347/2683] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:50:01 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:50:01.624876 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1214cc0e-47bc-4dd0-8139-0f086c9dba9c None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:01.685215 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1214cc0e-47bc-4dd0-8139-0f086c9dba9c None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:01.689850 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54448513-3a64-41e2-b069-2e510d12cf9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:01.690515 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54448513-3a64-41e2-b069-2e510d12cf9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:01.690792 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54448513-3a64-41e2-b069-2e510d12cf9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:01.691080 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54448513-3a64-41e2-b069-2e510d12cf9a admin admin] PATH_INFO: `/v3/users/5bfe97a0c772463baa010e62c4305793` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:01.692323 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54448513-3a64-41e2-b069-2e510d12cf9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bfe97a0c772463baa010e62c4305793)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:01.694396 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54448513-3a64-41e2-b069-2e510d12cf9a admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:01.730996 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:50:01.745666 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-54448513-3a64-41e2-b069-2e510d12cf9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bfe97a0c772463baa010e62c4305793'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:01.752590 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-54448513-3a64-41e2-b069-2e510d12cf9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bfe97a0c772463baa010e62c4305793'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:01.765084 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-54448513-3a64-41e2-b069-2e510d12cf9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bfe97a0c772463baa010e62c4305793'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:01.772511 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-54448513-3a64-41e2-b069-2e510d12cf9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bfe97a0c772463baa010e62c4305793'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:01.773155 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-54448513-3a64-41e2-b069-2e510d12cf9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bfe97a0c772463baa010e62c4305793'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:01.780174 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1337/2684] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:50:01 2022] DELETE /identity/v3/users/5bfe97a0c772463baa010e62c4305793 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:50:01.785850 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cc89a2-c02d-408b-aed5-992953a41e91 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:01.786135 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cc89a2-c02d-408b-aed5-992953a41e91 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:01.786444 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cc89a2-c02d-408b-aed5-992953a41e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:01.809474 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-00cc89a2-c02d-408b-aed5-992953a41e91 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:50:01.810802 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-00cc89a2-c02d-408b-aed5-992953a41e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:01.870052 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:50:01.880778 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1348/2685] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:50:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:50:01.893142 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54448513-3a64-41e2-b069-2e510d12cf9a admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:01.894807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-54448513-3a64-41e2-b069-2e510d12cf9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:01.953268 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54448513-3a64-41e2-b069-2e510d12cf9a admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:01.956892 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2579c543-6a8f-421c-b7cd-80201ff42423 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:01.957486 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2579c543-6a8f-421c-b7cd-80201ff42423 service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:01.957760 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2579c543-6a8f-421c-b7cd-80201ff42423 service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:01.958027 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2579c543-6a8f-421c-b7cd-80201ff42423 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:01.967234 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2579c543-6a8f-421c-b7cd-80201ff42423 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:01.968642 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2579c543-6a8f-421c-b7cd-80201ff42423 service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:01.978168 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1338/2686] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:50:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:50:02.262527 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cc89a2-c02d-408b-aed5-992953a41e91 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:02.264111 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-00cc89a2-c02d-408b-aed5-992953a41e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:02.283541 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cc89a2-c02d-408b-aed5-992953a41e91 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:02.289370 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017c8440-172e-4447-906b-98e08b8f00fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:02.290135 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017c8440-172e-4447-906b-98e08b8f00fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:02.290299 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017c8440-172e-4447-906b-98e08b8f00fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:02.290657 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017c8440-172e-4447-906b-98e08b8f00fe admin admin] PATH_INFO: `/v3/projects/98d16937467b4c98881dc45e6025e346` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:02.291767 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017c8440-172e-4447-906b-98e08b8f00fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=98d16937467b4c98881dc45e6025e346)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:02.293822 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017c8440-172e-4447-906b-98e08b8f00fe admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:02.349370 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-017c8440-172e-4447-906b-98e08b8f00fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98d16937467b4c98881dc45e6025e346', 'request_id': 'req-017c8440-172e-4447-906b-98e08b8f00fe'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:02.356856 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-017c8440-172e-4447-906b-98e08b8f00fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98d16937467b4c98881dc45e6025e346', 'request_id': 'req-017c8440-172e-4447-906b-98e08b8f00fe'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:02.362563 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-017c8440-172e-4447-906b-98e08b8f00fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98d16937467b4c98881dc45e6025e346', 'request_id': 'req-017c8440-172e-4447-906b-98e08b8f00fe'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:02.375679 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-017c8440-172e-4447-906b-98e08b8f00fe admin admin] The token cache is being invalidate because project 98d16937467b4c98881dc45e6025e346 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81617) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:50:02.375889 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-017c8440-172e-4447-906b-98e08b8f00fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:02.376514 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-017c8440-172e-4447-906b-98e08b8f00fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:02.377984 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1349/2687] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:50:02 2022] DELETE /identity/v3/projects/98d16937467b4c98881dc45e6025e346 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:50:02.384003 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ce658-7e2f-4c45-be30-f1ef5833cacc None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:02.384519 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ce658-7e2f-4c45-be30-f1ef5833cacc None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:02.384679 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ce658-7e2f-4c45-be30-f1ef5833cacc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:02.404460 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-fe4ce658-7e2f-4c45-be30-f1ef5833cacc None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:50:02.405707 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-fe4ce658-7e2f-4c45-be30-f1ef5833cacc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:02.471313 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1339/2688] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:50:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:50:02.473026 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b912ecbe-9864-4fc5-ad76-941734fd50d5 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:02.473334 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b912ecbe-9864-4fc5-ad76-941734fd50d5 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:02.473608 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b912ecbe-9864-4fc5-ad76-941734fd50d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:02.498876 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-b912ecbe-9864-4fc5-ad76-941734fd50d5 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:50:02.500114 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-b912ecbe-9864-4fc5-ad76-941734fd50d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:02.508243 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1350/2689] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:50:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:50:02.512730 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ce658-7e2f-4c45-be30-f1ef5833cacc None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:02.522262 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ce658-7e2f-4c45-be30-f1ef5833cacc None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:02.527399 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde3d603-5514-490b-9ea4-19fdd472d78f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:02.527983 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde3d603-5514-490b-9ea4-19fdd472d78f admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:02.528265 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde3d603-5514-490b-9ea4-19fdd472d78f admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:02.528534 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde3d603-5514-490b-9ea4-19fdd472d78f admin admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:02.529170 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde3d603-5514-490b-9ea4-19fdd472d78f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:50:02.529473 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde3d603-5514-490b-9ea4-19fdd472d78f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:02.531081 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde3d603-5514-490b-9ea4-19fdd472d78f admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:02.544386 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1340/2690] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:50:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:50:02.548621 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b912ecbe-9864-4fc5-ad76-941734fd50d5 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:02.560514 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b912ecbe-9864-4fc5-ad76-941734fd50d5 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:02.565948 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1602faeb-38f2-4327-b688-9f4cb19f68c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:02.566599 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1602faeb-38f2-4327-b688-9f4cb19f68c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:02.566878 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1602faeb-38f2-4327-b688-9f4cb19f68c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:02.567146 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1602faeb-38f2-4327-b688-9f4cb19f68c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:02.567797 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1602faeb-38f2-4327-b688-9f4cb19f68c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:02.570599 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1602faeb-38f2-4327-b688-9f4cb19f68c5 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:02.604134 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1351/2691] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:50:02 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:50:02.605116 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde3d603-5514-490b-9ea4-19fdd472d78f admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:02.616550 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde3d603-5514-490b-9ea4-19fdd472d78f admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:02.667220 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:50:02.688489 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd0b6fa-4c41-4150-8fa1-5aa1b8ccd0bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:02.689099 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd0b6fa-4c41-4150-8fa1-5aa1b8ccd0bf admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:02.689376 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd0b6fa-4c41-4150-8fa1-5aa1b8ccd0bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:02.689646 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd0b6fa-4c41-4150-8fa1-5aa1b8ccd0bf admin admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:02.690287 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd0b6fa-4c41-4150-8fa1-5aa1b8ccd0bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:02.692247 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd0b6fa-4c41-4150-8fa1-5aa1b8ccd0bf admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:02.714207 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1341/2692] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:50:02 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:50:02.718489 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1602faeb-38f2-4327-b688-9f4cb19f68c5 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:02.731073 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1602faeb-38f2-4327-b688-9f4cb19f68c5 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:02.735781 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21006cd-1cdf-49f2-808a-9d746ab5fb5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:02.736392 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21006cd-1cdf-49f2-808a-9d746ab5fb5c admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:02.736673 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21006cd-1cdf-49f2-808a-9d746ab5fb5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:02.736944 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21006cd-1cdf-49f2-808a-9d746ab5fb5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:02.737588 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21006cd-1cdf-49f2-808a-9d746ab5fb5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:02.739218 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21006cd-1cdf-49f2-808a-9d746ab5fb5c admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:02.750870 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1352/2693] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:50:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 16:50:02.755119 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd0b6fa-4c41-4150-8fa1-5aa1b8ccd0bf admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:02.765452 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd0b6fa-4c41-4150-8fa1-5aa1b8ccd0bf admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:02.770215 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eafbef-4708-4445-93c6-bcc7db43d518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:02.770843 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eafbef-4708-4445-93c6-bcc7db43d518 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:02.771120 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eafbef-4708-4445-93c6-bcc7db43d518 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:02.771391 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eafbef-4708-4445-93c6-bcc7db43d518 admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:02.772065 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4eafbef-4708-4445-93c6-bcc7db43d518 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:02.774201 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4eafbef-4708-4445-93c6-bcc7db43d518 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:02.784819 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1342/2694] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:50:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:50:02.789614 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21006cd-1cdf-49f2-808a-9d746ab5fb5c admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:02.800905 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21006cd-1cdf-49f2-808a-9d746ab5fb5c admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:02.806118 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115cc578-eeb3-454b-ac5a-6019b5fc426b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:02.807071 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115cc578-eeb3-454b-ac5a-6019b5fc426b admin admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:02.807354 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115cc578-eeb3-454b-ac5a-6019b5fc426b admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:02.807632 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115cc578-eeb3-454b-ac5a-6019b5fc426b admin admin] PATH_INFO: `/v3/projects/ce6e5f2a0d944d248c916552fde288da/users/3563f6e82d834ee4b4defcbe7fd363be/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:02.824904 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115cc578-eeb3-454b-ac5a-6019b5fc426b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce6e5f2a0d944d248c916552fde288da, user_id=3563f6e82d834ee4b4defcbe7fd363be, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:02.827630 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115cc578-eeb3-454b-ac5a-6019b5fc426b admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:02.843279 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1353/2695] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:50:02 2022] PUT /identity/v3/projects/ce6e5f2a0d944d248c916552fde288da/users/3563f6e82d834ee4b4defcbe7fd363be/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:50:02.855107 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0b7387-cfa6-4c08-bcb2-4cb362cfa34e None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:02.855439 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0b7387-cfa6-4c08-bcb2-4cb362cfa34e None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:02.855767 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0b7387-cfa6-4c08-bcb2-4cb362cfa34e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:02.888178 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-9e0b7387-cfa6-4c08-bcb2-4cb362cfa34e None None] MFA Rules not processed for user `3563f6e82d834ee4b4defcbe7fd363be`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:50:02.889825 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-9e0b7387-cfa6-4c08-bcb2-4cb362cfa34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:02.954759 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1343/2696] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:50:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:50:02.959543 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115cc578-eeb3-454b-ac5a-6019b5fc426b admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:03.020839 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115cc578-eeb3-454b-ac5a-6019b5fc426b admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:03.026758 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099393c2-92b1-42c9-9314-b95b262b1ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:03.027424 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099393c2-92b1-42c9-9314-b95b262b1ddd admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:03.027701 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099393c2-92b1-42c9-9314-b95b262b1ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:03.027993 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099393c2-92b1-42c9-9314-b95b262b1ddd admin admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:03.028671 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099393c2-92b1-42c9-9314-b95b262b1ddd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:03.030707 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099393c2-92b1-42c9-9314-b95b262b1ddd admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:03.061207 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1354/2697] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:50:02 2022] POST /identity/v3/projects => generated 385 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:50:03.065443 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0b7387-cfa6-4c08-bcb2-4cb362cfa34e None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:03.077150 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0b7387-cfa6-4c08-bcb2-4cb362cfa34e None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:03.135742 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb01165-85bd-491e-b375-f9728128b7b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:03.136351 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb01165-85bd-491e-b375-f9728128b7b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:03.136634 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb01165-85bd-491e-b375-f9728128b7b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:03.136908 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb01165-85bd-491e-b375-f9728128b7b3 admin admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:03.137553 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb01165-85bd-491e-b375-f9728128b7b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:03.139536 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb01165-85bd-491e-b375-f9728128b7b3 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:03.161439 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1344/2698] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:50:03 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:50:03.165786 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099393c2-92b1-42c9-9314-b95b262b1ddd admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:03.171390 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:50:03.177502 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099393c2-92b1-42c9-9314-b95b262b1ddd admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:03.183619 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e934a-6c9a-4d04-8e55-9c2fb850f1a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:03.184220 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e934a-6c9a-4d04-8e55-9c2fb850f1a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:03.184516 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e934a-6c9a-4d04-8e55-9c2fb850f1a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:03.184786 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e934a-6c9a-4d04-8e55-9c2fb850f1a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:03.185422 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19e934a-6c9a-4d04-8e55-9c2fb850f1a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:03.187037 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19e934a-6c9a-4d04-8e55-9c2fb850f1a0 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:03.198398 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1355/2699] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:50:03.202449 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb01165-85bd-491e-b375-f9728128b7b3 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:03.213081 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb01165-85bd-491e-b375-f9728128b7b3 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:03.218940 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dc8c96-4377-4385-8906-c3c625a670b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:03.219617 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dc8c96-4377-4385-8906-c3c625a670b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:03.219828 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dc8c96-4377-4385-8906-c3c625a670b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:03.220106 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dc8c96-4377-4385-8906-c3c625a670b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:03.220979 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dc8c96-4377-4385-8906-c3c625a670b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:03.223453 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dc8c96-4377-4385-8906-c3c625a670b0 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:03.235073 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1345/2700] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:50:03.239437 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e934a-6c9a-4d04-8e55-9c2fb850f1a0 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:03.250273 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e934a-6c9a-4d04-8e55-9c2fb850f1a0 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:03.256235 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125b4881-4389-4783-a565-49425f16cd3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:03.257209 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b4881-4389-4783-a565-49425f16cd3c admin admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:03.258147 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b4881-4389-4783-a565-49425f16cd3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:03.258551 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b4881-4389-4783-a565-49425f16cd3c admin admin] PATH_INFO: `/v3/projects/6db8edf887f746a39338320775b19f81/users/eaba5ef8860141c7a2be5f4ed5295ac1/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:03.274646 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125b4881-4389-4783-a565-49425f16cd3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6db8edf887f746a39338320775b19f81, user_id=eaba5ef8860141c7a2be5f4ed5295ac1, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:03.277405 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125b4881-4389-4783-a565-49425f16cd3c admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:03.294882 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1356/2701] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:50:03 2022] PUT /identity/v3/projects/6db8edf887f746a39338320775b19f81/users/eaba5ef8860141c7a2be5f4ed5295ac1/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:50:03.306130 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec9f27-e446-4662-899e-1f0dbc13e261 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:03.306797 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec9f27-e446-4662-899e-1f0dbc13e261 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:03.307145 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec9f27-e446-4662-899e-1f0dbc13e261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:03.343621 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-89ec9f27-e446-4662-899e-1f0dbc13e261 None None] MFA Rules not processed for user `eaba5ef8860141c7a2be5f4ed5295ac1`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:50:03.344847 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-89ec9f27-e446-4662-899e-1f0dbc13e261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:03.357884 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:50:03.417810 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1346/2702] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:50:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:50:03.422298 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125b4881-4389-4783-a565-49425f16cd3c admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:03.498720 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125b4881-4389-4783-a565-49425f16cd3c admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:03.505463 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c4a1a6-c25e-4a6e-8436-e9c0ebae4800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:03.506507 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c4a1a6-c25e-4a6e-8436-e9c0ebae4800 admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:03.506991 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c4a1a6-c25e-4a6e-8436-e9c0ebae4800 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:03.508142 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c4a1a6-c25e-4a6e-8436-e9c0ebae4800 admin admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:03.509384 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c4a1a6-c25e-4a6e-8436-e9c0ebae4800 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:03.513181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c4a1a6-c25e-4a6e-8436-e9c0ebae4800 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:03.564204 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1357/2703] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:50:03 2022] POST /identity/v3/projects => generated 385 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:50:03.565685 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ec9f27-e446-4662-899e-1f0dbc13e261 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:03.581967 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ec9f27-e446-4662-899e-1f0dbc13e261 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:03.647793 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60af924e-7e23-4909-9a68-601a2d1c17ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:03.648469 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60af924e-7e23-4909-9a68-601a2d1c17ba admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:03.648775 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60af924e-7e23-4909-9a68-601a2d1c17ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:03.649066 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60af924e-7e23-4909-9a68-601a2d1c17ba admin admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:03.649750 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60af924e-7e23-4909-9a68-601a2d1c17ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:03.651706 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60af924e-7e23-4909-9a68-601a2d1c17ba admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:03.679967 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1347/2704] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:50:03 2022] POST /identity/v3/users => generated 299 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:50:03.683884 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c4a1a6-c25e-4a6e-8436-e9c0ebae4800 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:03.695015 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c4a1a6-c25e-4a6e-8436-e9c0ebae4800 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:03.700457 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1095c177-0628-4ebf-81aa-a331d13b4629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:03.701071 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1095c177-0628-4ebf-81aa-a331d13b4629 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:03.701351 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1095c177-0628-4ebf-81aa-a331d13b4629 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:03.701626 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1095c177-0628-4ebf-81aa-a331d13b4629 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:03.702280 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1095c177-0628-4ebf-81aa-a331d13b4629 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:03.703910 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1095c177-0628-4ebf-81aa-a331d13b4629 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:03.719023 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1358/2705] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 16:50:03.723672 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60af924e-7e23-4909-9a68-601a2d1c17ba admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:03.750629 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60af924e-7e23-4909-9a68-601a2d1c17ba admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:03.750629 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1448db-ac9d-4104-b5bf-4a966aac36ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:03.750629 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1448db-ac9d-4104-b5bf-4a966aac36ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:03.750629 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1448db-ac9d-4104-b5bf-4a966aac36ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:03.751204 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1448db-ac9d-4104-b5bf-4a966aac36ee admin admin] PATH_INFO: `/v3/projects/6ecf0ba85b7a44f08eb351105a321eb4/users/1860757c82c648ccb21691d3fd1fd8ff/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:03.764894 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1448db-ac9d-4104-b5bf-4a966aac36ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ecf0ba85b7a44f08eb351105a321eb4, user_id=1860757c82c648ccb21691d3fd1fd8ff, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:03.767928 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1448db-ac9d-4104-b5bf-4a966aac36ee admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:03.786364 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1348/2706] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:50:03 2022] PUT /identity/v3/projects/6ecf0ba85b7a44f08eb351105a321eb4/users/1860757c82c648ccb21691d3fd1fd8ff/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:50:03.791011 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1095c177-0628-4ebf-81aa-a331d13b4629 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:03.801471 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1095c177-0628-4ebf-81aa-a331d13b4629 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:03.865024 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980c463e-470b-4532-9a01-634bccdcbc40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:03.865639 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980c463e-470b-4532-9a01-634bccdcbc40 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:03.865916 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980c463e-470b-4532-9a01-634bccdcbc40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:03.866190 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980c463e-470b-4532-9a01-634bccdcbc40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:03.867441 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980c463e-470b-4532-9a01-634bccdcbc40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:03.869088 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980c463e-470b-4532-9a01-634bccdcbc40 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:03.879799 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1359/2707] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:50:03.884173 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1448db-ac9d-4104-b5bf-4a966aac36ee admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:03.894776 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1448db-ac9d-4104-b5bf-4a966aac36ee admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:03.899531 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a430a3a-fc30-4677-9479-30b951dc6683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:03.900302 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a430a3a-fc30-4677-9479-30b951dc6683 admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:03.900665 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a430a3a-fc30-4677-9479-30b951dc6683 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:03.901002 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a430a3a-fc30-4677-9479-30b951dc6683 admin admin] PATH_INFO: `/v3/projects/6ecf0ba85b7a44f08eb351105a321eb4/users/1860757c82c648ccb21691d3fd1fd8ff/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:03.903164 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a430a3a-fc30-4677-9479-30b951dc6683 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ecf0ba85b7a44f08eb351105a321eb4, user_id=1860757c82c648ccb21691d3fd1fd8ff, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:03.905834 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a430a3a-fc30-4677-9479-30b951dc6683 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:03.918869 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1349/2708] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:50:03 2022] PUT /identity/v3/projects/6ecf0ba85b7a44f08eb351105a321eb4/users/1860757c82c648ccb21691d3fd1fd8ff/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:50:03.931183 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac10d9-22eb-46ce-a33d-b752514cd169 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:03.931410 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac10d9-22eb-46ce-a33d-b752514cd169 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:03.932000 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac10d9-22eb-46ce-a33d-b752514cd169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:03.962607 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-a8ac10d9-22eb-46ce-a33d-b752514cd169 None None] MFA Rules not processed for user `1860757c82c648ccb21691d3fd1fd8ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:50:03.963818 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-a8ac10d9-22eb-46ce-a33d-b752514cd169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:03.989942 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:50:04.037574 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1360/2709] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:50:03 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:50:04.049824 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a430a3a-fc30-4677-9479-30b951dc6683 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:04.051584 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-3a430a3a-fc30-4677-9479-30b951dc6683 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:04.118642 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a430a3a-fc30-4677-9479-30b951dc6683 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:04.125035 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c46939-9d67-4dbc-9ed1-53b7946a3554 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:04.125690 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c46939-9d67-4dbc-9ed1-53b7946a3554 service cinder] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:04.126157 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c46939-9d67-4dbc-9ed1-53b7946a3554 service cinder] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:04.126862 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c46939-9d67-4dbc-9ed1-53b7946a3554 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:04.153881 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:50:04.192266 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c46939-9d67-4dbc-9ed1-53b7946a3554 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:04.193605 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c46939-9d67-4dbc-9ed1-53b7946a3554 service cinder] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:04.202448 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1350/2710] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:50:04 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:50:05.790836 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ac10d9-22eb-46ce-a33d-b752514cd169 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:05.810813 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ac10d9-22eb-46ce-a33d-b752514cd169 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:05.838180 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19251fb4-8299-4a51-a9ae-455e4a5ca556 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:05.838848 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19251fb4-8299-4a51-a9ae-455e4a5ca556 service cinder] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:05.839132 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19251fb4-8299-4a51-a9ae-455e4a5ca556 service cinder] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:05.839416 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19251fb4-8299-4a51-a9ae-455e4a5ca556 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:05.905670 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19251fb4-8299-4a51-a9ae-455e4a5ca556 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:05.907144 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19251fb4-8299-4a51-a9ae-455e4a5ca556 service cinder] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:05.929336 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1361/2711] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:50:05 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 16:50:06.650525 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c46939-9d67-4dbc-9ed1-53b7946a3554 service cinder] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:06.659875 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c46939-9d67-4dbc-9ed1-53b7946a3554 service cinder] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:06.664254 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25fc7c8-6da8-45fd-aeac-de08507784c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:06.664866 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25fc7c8-6da8-45fd-aeac-de08507784c0 service cinder] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:06.665142 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25fc7c8-6da8-45fd-aeac-de08507784c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:06.665416 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25fc7c8-6da8-45fd-aeac-de08507784c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:06.675352 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25fc7c8-6da8-45fd-aeac-de08507784c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:06.676756 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25fc7c8-6da8-45fd-aeac-de08507784c0 service cinder] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:06.689347 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1351/2712] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:50:06 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 16:50:07.179668 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19251fb4-8299-4a51-a9ae-455e4a5ca556 service cinder] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:07.181462 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-19251fb4-8299-4a51-a9ae-455e4a5ca556 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:07.243795 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19251fb4-8299-4a51-a9ae-455e4a5ca556 service cinder] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:07.249232 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ed577e-d369-44d3-872f-2349a62360da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:07.249952 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ed577e-d369-44d3-872f-2349a62360da service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:07.250305 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ed577e-d369-44d3-872f-2349a62360da service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:07.250516 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ed577e-d369-44d3-872f-2349a62360da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:07.252310 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-21ed577e-d369-44d3-872f-2349a62360da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:07.264253 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ed577e-d369-44d3-872f-2349a62360da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:07.266011 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ed577e-d369-44d3-872f-2349a62360da service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:07.276855 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1362/2713] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:50:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:50:08.966527 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563458f5-41c2-4643-ab82-f3b1455c3f65 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:08.967020 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563458f5-41c2-4643-ab82-f3b1455c3f65 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:08.967136 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563458f5-41c2-4643-ab82-f3b1455c3f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:08.986776 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-563458f5-41c2-4643-ab82-f3b1455c3f65 None None] MFA Rules not processed for user `7222bd3b313b47e5a4fa0ebc9ff0cdd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:50:08.988063 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-563458f5-41c2-4643-ab82-f3b1455c3f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:09.050569 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1352/2714] 10.222.0.57 () {58 vars in 967 bytes} [Mon Sep 19 16:50:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:50:09.065254 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ed577e-d369-44d3-872f-2349a62360da service neutron] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:09.067075 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-21ed577e-d369-44d3-872f-2349a62360da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:09.145275 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ed577e-d369-44d3-872f-2349a62360da service neutron] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:09.151256 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73b481e-2108-41b1-b186-eb7cd1dfe2f3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '884a0524aca744b99c7496c11c3291ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:09.151883 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73b481e-2108-41b1-b186-eb7cd1dfe2f3 service swift] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:09.152164 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73b481e-2108-41b1-b186-eb7cd1dfe2f3 service swift] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:09.152441 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73b481e-2108-41b1-b186-eb7cd1dfe2f3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:09.157730 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:50:09.163137 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73b481e-2108-41b1-b186-eb7cd1dfe2f3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:09.164762 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73b481e-2108-41b1-b186-eb7cd1dfe2f3 service swift] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:09.174028 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1363/2715] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Sep 19 16:50:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 16:50:14.891588 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563458f5-41c2-4643-ab82-f3b1455c3f65 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:14.953085 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563458f5-41c2-4643-ab82-f3b1455c3f65 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:14.957897 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a416d73b-f40d-4691-b928-987165c4a9eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:14.958578 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a416d73b-f40d-4691-b928-987165c4a9eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:14.958866 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a416d73b-f40d-4691-b928-987165c4a9eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:14.959128 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a416d73b-f40d-4691-b928-987165c4a9eb admin admin] PATH_INFO: `/v3/users/3563f6e82d834ee4b4defcbe7fd363be` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:14.960263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a416d73b-f40d-4691-b928-987165c4a9eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=3563f6e82d834ee4b4defcbe7fd363be)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:14.962215 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a416d73b-f40d-4691-b928-987165c4a9eb admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:15.009869 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-a416d73b-f40d-4691-b928-987165c4a9eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3563f6e82d834ee4b4defcbe7fd363be'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:15.017361 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-a416d73b-f40d-4691-b928-987165c4a9eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3563f6e82d834ee4b4defcbe7fd363be'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:15.018659 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:50:15.034159 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-a416d73b-f40d-4691-b928-987165c4a9eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3563f6e82d834ee4b4defcbe7fd363be'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:15.056232 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-a416d73b-f40d-4691-b928-987165c4a9eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3563f6e82d834ee4b4defcbe7fd363be'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:15.057150 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-a416d73b-f40d-4691-b928-987165c4a9eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3563f6e82d834ee4b4defcbe7fd363be'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:15.063503 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1353/2716] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:50:14 2022] DELETE /identity/v3/users/3563f6e82d834ee4b4defcbe7fd363be => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:50:15.068071 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1ebe3b-6e57-43e6-9c24-e26fe5bdcc66 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:15.068601 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1ebe3b-6e57-43e6-9c24-e26fe5bdcc66 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:15.068709 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1ebe3b-6e57-43e6-9c24-e26fe5bdcc66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:15.089479 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-ee1ebe3b-6e57-43e6-9c24-e26fe5bdcc66 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:50:15.090892 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-ee1ebe3b-6e57-43e6-9c24-e26fe5bdcc66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:15.154692 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1364/2717] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:50:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:50:15.166309 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a416d73b-f40d-4691-b928-987165c4a9eb admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:15.167744 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-a416d73b-f40d-4691-b928-987165c4a9eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:15.224979 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a416d73b-f40d-4691-b928-987165c4a9eb admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:15.229181 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277748cf-9e85-44ca-bc09-e6fb31855e36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:15.229778 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277748cf-9e85-44ca-bc09-e6fb31855e36 service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:15.230060 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277748cf-9e85-44ca-bc09-e6fb31855e36 service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:15.230336 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277748cf-9e85-44ca-bc09-e6fb31855e36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:15.239237 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277748cf-9e85-44ca-bc09-e6fb31855e36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:15.240641 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277748cf-9e85-44ca-bc09-e6fb31855e36 service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:15.250251 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1354/2718] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:50:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:50:15.553390 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1ebe3b-6e57-43e6-9c24-e26fe5bdcc66 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:15.556031 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-ee1ebe3b-6e57-43e6-9c24-e26fe5bdcc66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:15.573457 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1ebe3b-6e57-43e6-9c24-e26fe5bdcc66 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:15.581430 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90f3fb0-3c58-478b-a8d3-c63410c21c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:15.582067 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f3fb0-3c58-478b-a8d3-c63410c21c16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:15.582340 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f3fb0-3c58-478b-a8d3-c63410c21c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:15.582641 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f3fb0-3c58-478b-a8d3-c63410c21c16 admin admin] PATH_INFO: `/v3/projects/ce6e5f2a0d944d248c916552fde288da` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:15.585298 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90f3fb0-3c58-478b-a8d3-c63410c21c16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce6e5f2a0d944d248c916552fde288da)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:15.587316 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90f3fb0-3c58-478b-a8d3-c63410c21c16 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:15.654549 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-a90f3fb0-3c58-478b-a8d3-c63410c21c16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce6e5f2a0d944d248c916552fde288da', 'request_id': 'req-a90f3fb0-3c58-478b-a8d3-c63410c21c16'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:15.670595 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-a90f3fb0-3c58-478b-a8d3-c63410c21c16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce6e5f2a0d944d248c916552fde288da', 'request_id': 'req-a90f3fb0-3c58-478b-a8d3-c63410c21c16'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:15.670595 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-a90f3fb0-3c58-478b-a8d3-c63410c21c16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce6e5f2a0d944d248c916552fde288da', 'request_id': 'req-a90f3fb0-3c58-478b-a8d3-c63410c21c16'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:15.680925 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-a90f3fb0-3c58-478b-a8d3-c63410c21c16 admin admin] The token cache is being invalidate because project ce6e5f2a0d944d248c916552fde288da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81617) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:50:15.681214 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-a90f3fb0-3c58-478b-a8d3-c63410c21c16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:15.682151 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-a90f3fb0-3c58-478b-a8d3-c63410c21c16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:15.683652 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1365/2719] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:50:15 2022] DELETE /identity/v3/projects/ce6e5f2a0d944d248c916552fde288da => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:50:15.687794 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277748cf-9e85-44ca-bc09-e6fb31855e36 service neutron] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:15.697660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277748cf-9e85-44ca-bc09-e6fb31855e36 service neutron] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:15.755892 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27103cf4-6371-4eb4-8b4b-f54eb8f8812d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:15.756520 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27103cf4-6371-4eb4-8b4b-f54eb8f8812d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:15.756795 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27103cf4-6371-4eb4-8b4b-f54eb8f8812d admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:15.757078 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27103cf4-6371-4eb4-8b4b-f54eb8f8812d admin admin] PATH_INFO: `/v3/users/eaba5ef8860141c7a2be5f4ed5295ac1` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:15.758352 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27103cf4-6371-4eb4-8b4b-f54eb8f8812d admin admin] RBAC: Authorizing `identity:delete_user(user_id=eaba5ef8860141c7a2be5f4ed5295ac1)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:15.760260 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27103cf4-6371-4eb4-8b4b-f54eb8f8812d admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:15.826817 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-27103cf4-6371-4eb4-8b4b-f54eb8f8812d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eaba5ef8860141c7a2be5f4ed5295ac1'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:15.837949 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-27103cf4-6371-4eb4-8b4b-f54eb8f8812d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eaba5ef8860141c7a2be5f4ed5295ac1'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:15.853780 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-27103cf4-6371-4eb4-8b4b-f54eb8f8812d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eaba5ef8860141c7a2be5f4ed5295ac1'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:15.859324 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:50:15.866875 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-27103cf4-6371-4eb4-8b4b-f54eb8f8812d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eaba5ef8860141c7a2be5f4ed5295ac1'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:15.866875 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-27103cf4-6371-4eb4-8b4b-f54eb8f8812d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eaba5ef8860141c7a2be5f4ed5295ac1'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:15.873532 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1355/2720] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:50:15 2022] DELETE /identity/v3/users/eaba5ef8860141c7a2be5f4ed5295ac1 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 16:50:16.151785 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90f3fb0-3c58-478b-a8d3-c63410c21c16 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:16.153262 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-a90f3fb0-3c58-478b-a8d3-c63410c21c16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:16.174657 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90f3fb0-3c58-478b-a8d3-c63410c21c16 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:16.233928 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002fe039-10cb-4f6f-b68d-4224d362d68d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:16.234596 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002fe039-10cb-4f6f-b68d-4224d362d68d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:16.234881 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002fe039-10cb-4f6f-b68d-4224d362d68d admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:16.235156 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002fe039-10cb-4f6f-b68d-4224d362d68d admin admin] PATH_INFO: `/v3/projects/6db8edf887f746a39338320775b19f81` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:16.236280 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002fe039-10cb-4f6f-b68d-4224d362d68d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6db8edf887f746a39338320775b19f81)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:16.238175 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002fe039-10cb-4f6f-b68d-4224d362d68d admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:16.258877 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:50:16.285477 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-002fe039-10cb-4f6f-b68d-4224d362d68d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6db8edf887f746a39338320775b19f81', 'request_id': 'req-002fe039-10cb-4f6f-b68d-4224d362d68d'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:16.292441 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-002fe039-10cb-4f6f-b68d-4224d362d68d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6db8edf887f746a39338320775b19f81', 'request_id': 'req-002fe039-10cb-4f6f-b68d-4224d362d68d'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:16.298701 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-002fe039-10cb-4f6f-b68d-4224d362d68d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6db8edf887f746a39338320775b19f81', 'request_id': 'req-002fe039-10cb-4f6f-b68d-4224d362d68d'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:16.311541 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-002fe039-10cb-4f6f-b68d-4224d362d68d admin admin] The token cache is being invalidate because project 6db8edf887f746a39338320775b19f81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81617) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:50:16.311831 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-002fe039-10cb-4f6f-b68d-4224d362d68d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:16.312495 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-002fe039-10cb-4f6f-b68d-4224d362d68d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:16.314014 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1366/2721] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:50:16 2022] DELETE /identity/v3/projects/6db8edf887f746a39338320775b19f81 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:50:16.318333 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27103cf4-6371-4eb4-8b4b-f54eb8f8812d admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:16.319780 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-27103cf4-6371-4eb4-8b4b-f54eb8f8812d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:16.333381 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27103cf4-6371-4eb4-8b4b-f54eb8f8812d admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:16.389151 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec53ef7d-8c23-42b5-a493-7156e5e06259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:16.389779 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec53ef7d-8c23-42b5-a493-7156e5e06259 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:16.390060 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec53ef7d-8c23-42b5-a493-7156e5e06259 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:16.390333 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec53ef7d-8c23-42b5-a493-7156e5e06259 admin admin] PATH_INFO: `/v3/users/1860757c82c648ccb21691d3fd1fd8ff` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:16.391978 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec53ef7d-8c23-42b5-a493-7156e5e06259 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1860757c82c648ccb21691d3fd1fd8ff)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:16.393936 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec53ef7d-8c23-42b5-a493-7156e5e06259 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:16.444154 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-ec53ef7d-8c23-42b5-a493-7156e5e06259 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1860757c82c648ccb21691d3fd1fd8ff'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:16.451511 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-ec53ef7d-8c23-42b5-a493-7156e5e06259 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1860757c82c648ccb21691d3fd1fd8ff'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:16.464186 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-ec53ef7d-8c23-42b5-a493-7156e5e06259 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1860757c82c648ccb21691d3fd1fd8ff'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:16.471598 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-ec53ef7d-8c23-42b5-a493-7156e5e06259 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1860757c82c648ccb21691d3fd1fd8ff'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:16.472375 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-ec53ef7d-8c23-42b5-a493-7156e5e06259 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1860757c82c648ccb21691d3fd1fd8ff'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:16.478701 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1356/2722] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:50:16 2022] DELETE /identity/v3/users/1860757c82c648ccb21691d3fd1fd8ff => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:50:16.752331 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002fe039-10cb-4f6f-b68d-4224d362d68d admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:16.753756 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-002fe039-10cb-4f6f-b68d-4224d362d68d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:16.769757 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002fe039-10cb-4f6f-b68d-4224d362d68d admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:16.833036 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838bf05c-ec74-4cbf-ac18-6be17e9d7f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:16.833674 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838bf05c-ec74-4cbf-ac18-6be17e9d7f1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:16.833953 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838bf05c-ec74-4cbf-ac18-6be17e9d7f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:16.834226 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838bf05c-ec74-4cbf-ac18-6be17e9d7f1e admin admin] PATH_INFO: `/v3/projects/6ecf0ba85b7a44f08eb351105a321eb4` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:16.835390 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838bf05c-ec74-4cbf-ac18-6be17e9d7f1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ecf0ba85b7a44f08eb351105a321eb4)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:16.837826 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838bf05c-ec74-4cbf-ac18-6be17e9d7f1e admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:16.897610 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-838bf05c-ec74-4cbf-ac18-6be17e9d7f1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ecf0ba85b7a44f08eb351105a321eb4', 'request_id': 'req-838bf05c-ec74-4cbf-ac18-6be17e9d7f1e'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:16.905741 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-838bf05c-ec74-4cbf-ac18-6be17e9d7f1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ecf0ba85b7a44f08eb351105a321eb4', 'request_id': 'req-838bf05c-ec74-4cbf-ac18-6be17e9d7f1e'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:16.912633 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-838bf05c-ec74-4cbf-ac18-6be17e9d7f1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ecf0ba85b7a44f08eb351105a321eb4', 'request_id': 'req-838bf05c-ec74-4cbf-ac18-6be17e9d7f1e'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:16.934652 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-838bf05c-ec74-4cbf-ac18-6be17e9d7f1e admin admin] The token cache is being invalidate because project 6ecf0ba85b7a44f08eb351105a321eb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81617) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:50:16.934652 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-838bf05c-ec74-4cbf-ac18-6be17e9d7f1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:16.934652 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-838bf05c-ec74-4cbf-ac18-6be17e9d7f1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:16.934652 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1367/2723] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:50:16 2022] DELETE /identity/v3/projects/6ecf0ba85b7a44f08eb351105a321eb4 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:50:16.935062 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d1ecd-93e3-4ff2-809b-72f3e58da5d2 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:16.935062 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d1ecd-93e3-4ff2-809b-72f3e58da5d2 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:16.935257 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d1ecd-93e3-4ff2-809b-72f3e58da5d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:16.963845 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-f79d1ecd-93e3-4ff2-809b-72f3e58da5d2 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:50:16.965060 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-f79d1ecd-93e3-4ff2-809b-72f3e58da5d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:17.030858 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1357/2724] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:50:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:50:17.038486 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f24a497-02db-4f56-82d7-0a80f9679d91 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:17.038846 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f24a497-02db-4f56-82d7-0a80f9679d91 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:17.039809 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f24a497-02db-4f56-82d7-0a80f9679d91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:17.061844 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-6f24a497-02db-4f56-82d7-0a80f9679d91 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:50:17.063095 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-6f24a497-02db-4f56-82d7-0a80f9679d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:17.072188 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1368/2725] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:50:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:50:17.076724 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79d1ecd-93e3-4ff2-809b-72f3e58da5d2 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:17.087834 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79d1ecd-93e3-4ff2-809b-72f3e58da5d2 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:17.092676 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa93abd6-70e8-4fd6-a315-be85d32d271b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:17.093285 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa93abd6-70e8-4fd6-a315-be85d32d271b admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:17.093567 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa93abd6-70e8-4fd6-a315-be85d32d271b admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:17.093876 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa93abd6-70e8-4fd6-a315-be85d32d271b admin admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:17.094565 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa93abd6-70e8-4fd6-a315-be85d32d271b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:50:17.094875 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa93abd6-70e8-4fd6-a315-be85d32d271b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:17.096620 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa93abd6-70e8-4fd6-a315-be85d32d271b admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:17.108967 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1358/2726] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:50:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:50:17.113382 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f24a497-02db-4f56-82d7-0a80f9679d91 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:17.122635 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f24a497-02db-4f56-82d7-0a80f9679d91 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:17.127333 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7c9933-598e-49fb-a4ff-8df5a05eb894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:17.127922 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7c9933-598e-49fb-a4ff-8df5a05eb894 admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:17.128199 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7c9933-598e-49fb-a4ff-8df5a05eb894 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:17.128468 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7c9933-598e-49fb-a4ff-8df5a05eb894 admin admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:17.129092 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7c9933-598e-49fb-a4ff-8df5a05eb894 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:17.131023 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7c9933-598e-49fb-a4ff-8df5a05eb894 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:17.158364 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1369/2727] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:50:17 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:50:17.162575 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa93abd6-70e8-4fd6-a315-be85d32d271b admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:17.167733 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:50:17.173949 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa93abd6-70e8-4fd6-a315-be85d32d271b admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:17.231499 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feb5ddd-a53e-4621-a5c2-e82a690e614f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:17.232103 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feb5ddd-a53e-4621-a5c2-e82a690e614f admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:17.232385 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feb5ddd-a53e-4621-a5c2-e82a690e614f admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:17.232656 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feb5ddd-a53e-4621-a5c2-e82a690e614f admin admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:17.233276 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feb5ddd-a53e-4621-a5c2-e82a690e614f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:17.235192 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feb5ddd-a53e-4621-a5c2-e82a690e614f admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:17.256489 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1359/2728] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:50:17 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:50:17.260970 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7c9933-598e-49fb-a4ff-8df5a05eb894 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:17.271503 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7c9933-598e-49fb-a4ff-8df5a05eb894 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:17.276844 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81a31c9-750b-4874-ad7d-557a3bd632d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:17.277448 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a31c9-750b-4874-ad7d-557a3bd632d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:17.277730 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a31c9-750b-4874-ad7d-557a3bd632d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:17.278002 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a31c9-750b-4874-ad7d-557a3bd632d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:17.278681 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81a31c9-750b-4874-ad7d-557a3bd632d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:17.280421 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81a31c9-750b-4874-ad7d-557a3bd632d3 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:17.283381 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:50:17.293560 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1370/2729] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:50:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:50:17.298302 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feb5ddd-a53e-4621-a5c2-e82a690e614f admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:17.309387 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feb5ddd-a53e-4621-a5c2-e82a690e614f admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:17.314192 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15381da6-fc37-4e64-8d6a-95f2e7bcb1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:17.314821 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15381da6-fc37-4e64-8d6a-95f2e7bcb1a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:17.315128 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15381da6-fc37-4e64-8d6a-95f2e7bcb1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:17.315398 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15381da6-fc37-4e64-8d6a-95f2e7bcb1a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:17.316032 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15381da6-fc37-4e64-8d6a-95f2e7bcb1a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:17.317651 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15381da6-fc37-4e64-8d6a-95f2e7bcb1a9 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:17.328791 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1360/2730] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:50:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:50:17.333105 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81a31c9-750b-4874-ad7d-557a3bd632d3 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:17.343665 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81a31c9-750b-4874-ad7d-557a3bd632d3 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:17.348262 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3c08ca-3d46-471c-a923-456e3e8cfc45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:17.348900 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3c08ca-3d46-471c-a923-456e3e8cfc45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:17.349184 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3c08ca-3d46-471c-a923-456e3e8cfc45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:17.349460 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3c08ca-3d46-471c-a923-456e3e8cfc45 admin admin] PATH_INFO: `/v3/projects/9026e39904fa45eb9984dc3d687e7a08/users/c4bfbedbe6cd494984e181a29c36ed8c/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:17.363562 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3c08ca-3d46-471c-a923-456e3e8cfc45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9026e39904fa45eb9984dc3d687e7a08, user_id=c4bfbedbe6cd494984e181a29c36ed8c, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:17.366189 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3c08ca-3d46-471c-a923-456e3e8cfc45 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:17.379400 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1371/2731] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:50:17 2022] PUT /identity/v3/projects/9026e39904fa45eb9984dc3d687e7a08/users/c4bfbedbe6cd494984e181a29c36ed8c/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:50:17.390143 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b796508f-b4ab-49cb-b756-7bbda194cd94 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:17.390450 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b796508f-b4ab-49cb-b756-7bbda194cd94 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:17.390724 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b796508f-b4ab-49cb-b756-7bbda194cd94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:17.418592 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-b796508f-b4ab-49cb-b756-7bbda194cd94 None None] MFA Rules not processed for user `c4bfbedbe6cd494984e181a29c36ed8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:50:17.419807 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-b796508f-b4ab-49cb-b756-7bbda194cd94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:17.483270 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1361/2732] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:50:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:50:17.498642 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3c08ca-3d46-471c-a923-456e3e8cfc45 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:17.500357 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-cc3c08ca-3d46-471c-a923-456e3e8cfc45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:17.556476 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3c08ca-3d46-471c-a923-456e3e8cfc45 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:17.560346 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942d4fa9-f658-43e1-b4da-5d9371b6d316 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:17.560946 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d4fa9-f658-43e1-b4da-5d9371b6d316 service cinder] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:17.561224 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d4fa9-f658-43e1-b4da-5d9371b6d316 service cinder] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:17.561490 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d4fa9-f658-43e1-b4da-5d9371b6d316 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:17.569661 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942d4fa9-f658-43e1-b4da-5d9371b6d316 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:17.571110 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942d4fa9-f658-43e1-b4da-5d9371b6d316 service cinder] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:17.579707 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1372/2733] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:50:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:50:21.047002 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2d7080-fb0e-4552-b5a5-0f9bc564d384 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:21.047525 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2d7080-fb0e-4552-b5a5-0f9bc564d384 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:21.047525 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2d7080-fb0e-4552-b5a5-0f9bc564d384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:21.067225 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-ea2d7080-fb0e-4552-b5a5-0f9bc564d384 None None] MFA Rules not processed for user `c4bfbedbe6cd494984e181a29c36ed8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:50:21.068426 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-ea2d7080-fb0e-4552-b5a5-0f9bc564d384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:21.075331 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1362/2734] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:50:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:50:27.575787 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=37,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:50:31.062384 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=68,keystone:INSERT=3 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:50:34.973074 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942d4fa9-f658-43e1-b4da-5d9371b6d316 service cinder] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:34.974527 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-942d4fa9-f658-43e1-b4da-5d9371b6d316 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:35.041680 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942d4fa9-f658-43e1-b4da-5d9371b6d316 service cinder] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:35.046264 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810d4996-b257-461f-bc99-e14a2458f082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:35.046925 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d4996-b257-461f-bc99-e14a2458f082 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:35.047205 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d4996-b257-461f-bc99-e14a2458f082 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:35.047492 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d4996-b257-461f-bc99-e14a2458f082 admin admin] PATH_INFO: `/v3/users/9164b586372c4a9c82463c860501012e` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:35.048640 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810d4996-b257-461f-bc99-e14a2458f082 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9164b586372c4a9c82463c860501012e)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:35.050596 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810d4996-b257-461f-bc99-e14a2458f082 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:35.101211 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-810d4996-b257-461f-bc99-e14a2458f082 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9164b586372c4a9c82463c860501012e'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:35.101878 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-810d4996-b257-461f-bc99-e14a2458f082 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9164b586372c4a9c82463c860501012e'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:35.108989 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-810d4996-b257-461f-bc99-e14a2458f082 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9164b586372c4a9c82463c860501012e'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:35.121665 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-810d4996-b257-461f-bc99-e14a2458f082 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9164b586372c4a9c82463c860501012e'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:35.122512 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-810d4996-b257-461f-bc99-e14a2458f082 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9164b586372c4a9c82463c860501012e'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:35.137635 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1373/2735] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:50:34 2022] DELETE /identity/v3/users/9164b586372c4a9c82463c860501012e => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:50:35.142061 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5074c2a3-6ed3-4c5a-922c-c57703d9c3a7 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:35.142577 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5074c2a3-6ed3-4c5a-922c-c57703d9c3a7 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:35.142784 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5074c2a3-6ed3-4c5a-922c-c57703d9c3a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:35.162070 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-5074c2a3-6ed3-4c5a-922c-c57703d9c3a7 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:50:35.163306 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-5074c2a3-6ed3-4c5a-922c-c57703d9c3a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:35.230879 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1363/2736] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:50:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:50:35.240450 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810d4996-b257-461f-bc99-e14a2458f082 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:35.241862 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-810d4996-b257-461f-bc99-e14a2458f082 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:35.299449 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810d4996-b257-461f-bc99-e14a2458f082 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:35.303014 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7365ef4f-9ca1-4826-a979-297de1d5c29c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:35.303613 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7365ef4f-9ca1-4826-a979-297de1d5c29c service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:35.303886 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7365ef4f-9ca1-4826-a979-297de1d5c29c service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:35.304154 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7365ef4f-9ca1-4826-a979-297de1d5c29c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:35.313219 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7365ef4f-9ca1-4826-a979-297de1d5c29c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:35.314670 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7365ef4f-9ca1-4826-a979-297de1d5c29c service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:35.324568 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1374/2737] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:50:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:50:35.592561 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5074c2a3-6ed3-4c5a-922c-c57703d9c3a7 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:35.594655 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-5074c2a3-6ed3-4c5a-922c-c57703d9c3a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:35.614568 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5074c2a3-6ed3-4c5a-922c-c57703d9c3a7 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:35.638168 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1cf7ef-e58d-494a-af85-cfd11d533a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:35.638926 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1cf7ef-e58d-494a-af85-cfd11d533a71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:35.639326 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1cf7ef-e58d-494a-af85-cfd11d533a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:35.639664 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1cf7ef-e58d-494a-af85-cfd11d533a71 admin admin] PATH_INFO: `/v3/projects/dd0170949e9d4e5cb1ce092558d970ee` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:35.641450 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1cf7ef-e58d-494a-af85-cfd11d533a71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd0170949e9d4e5cb1ce092558d970ee)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:35.646989 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1cf7ef-e58d-494a-af85-cfd11d533a71 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:35.727630 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-6a1cf7ef-e58d-494a-af85-cfd11d533a71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd0170949e9d4e5cb1ce092558d970ee', 'request_id': 'req-6a1cf7ef-e58d-494a-af85-cfd11d533a71'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:35.734323 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-6a1cf7ef-e58d-494a-af85-cfd11d533a71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd0170949e9d4e5cb1ce092558d970ee', 'request_id': 'req-6a1cf7ef-e58d-494a-af85-cfd11d533a71'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:35.745291 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-6a1cf7ef-e58d-494a-af85-cfd11d533a71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd0170949e9d4e5cb1ce092558d970ee', 'request_id': 'req-6a1cf7ef-e58d-494a-af85-cfd11d533a71'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:35.761803 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-6a1cf7ef-e58d-494a-af85-cfd11d533a71 admin admin] The token cache is being invalidate because project dd0170949e9d4e5cb1ce092558d970ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81618) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:50:35.762213 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-6a1cf7ef-e58d-494a-af85-cfd11d533a71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:35.763637 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-6a1cf7ef-e58d-494a-af85-cfd11d533a71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:35.768587 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1364/2738] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:50:35 2022] DELETE /identity/v3/projects/dd0170949e9d4e5cb1ce092558d970ee => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:50:35.771509 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b97ba-3deb-4533-8c37-5c43bd30db9a None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:35.771797 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b97ba-3deb-4533-8c37-5c43bd30db9a None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:35.772166 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b97ba-3deb-4533-8c37-5c43bd30db9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:35.798802 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-529b97ba-3deb-4533-8c37-5c43bd30db9a None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:50:35.800058 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-529b97ba-3deb-4533-8c37-5c43bd30db9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:35.823588 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:50:35.872661 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1375/2739] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:50:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:50:35.880586 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1e2712-d512-4958-a365-ff0e9e078e89 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:35.880876 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1e2712-d512-4958-a365-ff0e9e078e89 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:35.881148 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1e2712-d512-4958-a365-ff0e9e078e89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:35.902585 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-fc1e2712-d512-4958-a365-ff0e9e078e89 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:50:35.903803 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-fc1e2712-d512-4958-a365-ff0e9e078e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:35.912085 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1365/2740] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:50:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:50:35.916950 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529b97ba-3deb-4533-8c37-5c43bd30db9a None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:35.926872 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529b97ba-3deb-4533-8c37-5c43bd30db9a None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:35.932241 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69847c8-88c3-4ef4-8909-6291a879ebf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:35.932834 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69847c8-88c3-4ef4-8909-6291a879ebf1 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:35.933114 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69847c8-88c3-4ef4-8909-6291a879ebf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:35.933387 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69847c8-88c3-4ef4-8909-6291a879ebf1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:35.934028 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69847c8-88c3-4ef4-8909-6291a879ebf1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:50:35.934333 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69847c8-88c3-4ef4-8909-6291a879ebf1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:35.935964 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69847c8-88c3-4ef4-8909-6291a879ebf1 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:35.947539 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1376/2741] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:50:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:50:35.951812 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1e2712-d512-4958-a365-ff0e9e078e89 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:35.961050 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1e2712-d512-4958-a365-ff0e9e078e89 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:35.965727 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb88548d-9fea-4156-be5c-09f36195114a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:35.966313 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb88548d-9fea-4156-be5c-09f36195114a admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:35.966628 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb88548d-9fea-4156-be5c-09f36195114a admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:35.966904 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb88548d-9fea-4156-be5c-09f36195114a admin admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:35.967543 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb88548d-9fea-4156-be5c-09f36195114a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:35.969415 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb88548d-9fea-4156-be5c-09f36195114a admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:36.002679 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1366/2742] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:50:35 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:50:36.003561 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69847c8-88c3-4ef4-8909-6291a879ebf1 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:36.013656 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69847c8-88c3-4ef4-8909-6291a879ebf1 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:36.076807 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73affbde-dfec-4617-8e9c-79686329dda5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:36.077570 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73affbde-dfec-4617-8e9c-79686329dda5 admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:36.077703 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73affbde-dfec-4617-8e9c-79686329dda5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:36.078058 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73affbde-dfec-4617-8e9c-79686329dda5 admin admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:36.078804 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73affbde-dfec-4617-8e9c-79686329dda5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:36.080817 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73affbde-dfec-4617-8e9c-79686329dda5 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:36.112106 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1377/2743] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:50:36 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 16:50:36.112239 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb88548d-9fea-4156-be5c-09f36195114a admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:36.120215 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb88548d-9fea-4156-be5c-09f36195114a admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:36.124909 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9aada7-5bd4-471d-99c5-7899cbc1d32c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:36.125507 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9aada7-5bd4-471d-99c5-7899cbc1d32c admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:36.125787 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9aada7-5bd4-471d-99c5-7899cbc1d32c admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:36.126059 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9aada7-5bd4-471d-99c5-7899cbc1d32c admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:36.126729 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9aada7-5bd4-471d-99c5-7899cbc1d32c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:36.128301 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9aada7-5bd4-471d-99c5-7899cbc1d32c admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:36.138881 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1367/2744] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:50:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 16:50:36.142814 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73affbde-dfec-4617-8e9c-79686329dda5 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:36.153171 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73affbde-dfec-4617-8e9c-79686329dda5 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:36.157758 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c8995b-16d2-4dad-899d-ec4910780d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:36.158351 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c8995b-16d2-4dad-899d-ec4910780d93 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:36.158658 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c8995b-16d2-4dad-899d-ec4910780d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:36.158934 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c8995b-16d2-4dad-899d-ec4910780d93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:36.159571 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c8995b-16d2-4dad-899d-ec4910780d93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:36.161153 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c8995b-16d2-4dad-899d-ec4910780d93 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:36.172264 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1378/2745] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:50:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:50:36.176295 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9aada7-5bd4-471d-99c5-7899cbc1d32c admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:36.186593 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9aada7-5bd4-471d-99c5-7899cbc1d32c admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:36.191483 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4346161-2be4-445d-8e7b-e1a5fb552eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:36.192124 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4346161-2be4-445d-8e7b-e1a5fb552eca admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:36.192403 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4346161-2be4-445d-8e7b-e1a5fb552eca admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:36.192680 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4346161-2be4-445d-8e7b-e1a5fb552eca admin admin] PATH_INFO: `/v3/projects/97472977854e45289f84d6732643d0f8/users/7c54c499b6aa488b81548d6231c5f3e6/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:36.207334 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4346161-2be4-445d-8e7b-e1a5fb552eca admin admin] RBAC: Authorizing `identity:create_grant(project_id=97472977854e45289f84d6732643d0f8, user_id=7c54c499b6aa488b81548d6231c5f3e6, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:36.209984 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4346161-2be4-445d-8e7b-e1a5fb552eca admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:36.223307 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1368/2746] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:50:36 2022] PUT /identity/v3/projects/97472977854e45289f84d6732643d0f8/users/7c54c499b6aa488b81548d6231c5f3e6/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 16:50:36.294149 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ef59cd-7e70-4051-bf04-bec013bf09ba None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:36.294639 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ef59cd-7e70-4051-bf04-bec013bf09ba None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:36.295028 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ef59cd-7e70-4051-bf04-bec013bf09ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:36.328778 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-46ef59cd-7e70-4051-bf04-bec013bf09ba None None] MFA Rules not processed for user `7c54c499b6aa488b81548d6231c5f3e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:50:36.329998 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-46ef59cd-7e70-4051-bf04-bec013bf09ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:36.394536 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1379/2747] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:50:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:50:36.408471 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4346161-2be4-445d-8e7b-e1a5fb552eca admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:36.410190 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-e4346161-2be4-445d-8e7b-e1a5fb552eca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:36.433164 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:50:36.472168 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4346161-2be4-445d-8e7b-e1a5fb552eca admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:36.475848 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7b03d9-63f1-4e0e-b015-e670c3326412 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:36.476449 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b03d9-63f1-4e0e-b015-e670c3326412 service cinder] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:36.476728 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b03d9-63f1-4e0e-b015-e670c3326412 service cinder] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:36.477003 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b03d9-63f1-4e0e-b015-e670c3326412 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:36.485661 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7b03d9-63f1-4e0e-b015-e670c3326412 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:36.487110 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7b03d9-63f1-4e0e-b015-e670c3326412 service cinder] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:36.495715 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1369/2748] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:50:36 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:50:38.277828 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ef59cd-7e70-4051-bf04-bec013bf09ba None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:38.280283 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-46ef59cd-7e70-4051-bf04-bec013bf09ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:38.303323 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:50:38.348335 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ef59cd-7e70-4051-bf04-bec013bf09ba None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:38.353864 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266efa3a-13a8-4dac-9041-990944bc728b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:38.355142 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266efa3a-13a8-4dac-9041-990944bc728b service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:38.356144 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266efa3a-13a8-4dac-9041-990944bc728b service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:38.356144 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266efa3a-13a8-4dac-9041-990944bc728b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:38.357782 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-266efa3a-13a8-4dac-9041-990944bc728b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:38.426016 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266efa3a-13a8-4dac-9041-990944bc728b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:38.427547 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266efa3a-13a8-4dac-9041-990944bc728b service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:38.438506 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1380/2749] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:50:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:50:38.952468 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7b03d9-63f1-4e0e-b015-e670c3326412 service cinder] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:38.953939 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-3c7b03d9-63f1-4e0e-b015-e670c3326412 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:38.969302 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7b03d9-63f1-4e0e-b015-e670c3326412 service cinder] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:38.973827 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0f2e21-c516-47a0-afb2-143f76344c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:38.974501 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0f2e21-c516-47a0-afb2-143f76344c4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:38.974782 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0f2e21-c516-47a0-afb2-143f76344c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:38.975080 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0f2e21-c516-47a0-afb2-143f76344c4b admin admin] PATH_INFO: `/v3/users/e1c0b2cfaa9f485f9116e012c62120fb` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:38.976196 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0f2e21-c516-47a0-afb2-143f76344c4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1c0b2cfaa9f485f9116e012c62120fb)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:38.978084 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0f2e21-c516-47a0-afb2-143f76344c4b admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:39.032124 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-9a0f2e21-c516-47a0-afb2-143f76344c4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1c0b2cfaa9f485f9116e012c62120fb'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:39.040670 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-9a0f2e21-c516-47a0-afb2-143f76344c4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1c0b2cfaa9f485f9116e012c62120fb'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:39.053397 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-9a0f2e21-c516-47a0-afb2-143f76344c4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1c0b2cfaa9f485f9116e012c62120fb'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:39.061370 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-9a0f2e21-c516-47a0-afb2-143f76344c4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1c0b2cfaa9f485f9116e012c62120fb'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:39.062025 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-9a0f2e21-c516-47a0-afb2-143f76344c4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1c0b2cfaa9f485f9116e012c62120fb'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:39.066405 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266efa3a-13a8-4dac-9041-990944bc728b service neutron] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:39.071440 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1370/2750] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:50:38 2022] DELETE /identity/v3/users/e1c0b2cfaa9f485f9116e012c62120fb => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:50:39.079970 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266efa3a-13a8-4dac-9041-990944bc728b service neutron] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:39.141057 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c726b042-6835-4280-a7f4-c08627976200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:39.141605 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c726b042-6835-4280-a7f4-c08627976200 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:39.141885 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c726b042-6835-4280-a7f4-c08627976200 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:39.142164 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c726b042-6835-4280-a7f4-c08627976200 admin admin] PATH_INFO: `/v3/users/7c54c499b6aa488b81548d6231c5f3e6` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:39.143349 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c726b042-6835-4280-a7f4-c08627976200 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c54c499b6aa488b81548d6231c5f3e6)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:39.145246 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c726b042-6835-4280-a7f4-c08627976200 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:39.194176 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-c726b042-6835-4280-a7f4-c08627976200 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c54c499b6aa488b81548d6231c5f3e6'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:39.194880 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-c726b042-6835-4280-a7f4-c08627976200 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c54c499b6aa488b81548d6231c5f3e6'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:39.203111 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-c726b042-6835-4280-a7f4-c08627976200 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c54c499b6aa488b81548d6231c5f3e6'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:39.213895 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:50:39.216757 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-c726b042-6835-4280-a7f4-c08627976200 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c54c499b6aa488b81548d6231c5f3e6'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:39.217731 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-c726b042-6835-4280-a7f4-c08627976200 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c54c499b6aa488b81548d6231c5f3e6'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:39.234660 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1381/2751] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:50:39 2022] DELETE /identity/v3/users/7c54c499b6aa488b81548d6231c5f3e6 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 16:50:39.239186 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd6b8d-8345-460c-bcba-c054b223a789 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:39.239513 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd6b8d-8345-460c-bcba-c054b223a789 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:39.239856 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd6b8d-8345-460c-bcba-c054b223a789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:39.251967 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c726b042-6835-4280-a7f4-c08627976200 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:39.253491 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-c726b042-6835-4280-a7f4-c08627976200 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:39.269925 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-06cd6b8d-8345-460c-bcba-c054b223a789 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:50:39.271699 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-06cd6b8d-8345-460c-bcba-c054b223a789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:39.326730 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c726b042-6835-4280-a7f4-c08627976200 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:39.331820 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80676137-8f35-45ea-8153-3f028c595ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:39.332575 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80676137-8f35-45ea-8153-3f028c595ce8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:39.332953 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80676137-8f35-45ea-8153-3f028c595ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:39.333320 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80676137-8f35-45ea-8153-3f028c595ce8 admin admin] PATH_INFO: `/v3/projects/5fc6461a081c479fa20c9fb15a2b1575` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:39.335303 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80676137-8f35-45ea-8153-3f028c595ce8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fc6461a081c479fa20c9fb15a2b1575)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:39.338326 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80676137-8f35-45ea-8153-3f028c595ce8 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:39.342077 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1371/2752] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:50:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:50:39.351611 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cd6b8d-8345-460c-bcba-c054b223a789 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:39.353625 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-06cd6b8d-8345-460c-bcba-c054b223a789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:39.389441 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-80676137-8f35-45ea-8153-3f028c595ce8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fc6461a081c479fa20c9fb15a2b1575', 'request_id': 'req-80676137-8f35-45ea-8153-3f028c595ce8'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:39.398107 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-80676137-8f35-45ea-8153-3f028c595ce8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fc6461a081c479fa20c9fb15a2b1575', 'request_id': 'req-80676137-8f35-45ea-8153-3f028c595ce8'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:39.401808 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:50:39.403775 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-80676137-8f35-45ea-8153-3f028c595ce8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fc6461a081c479fa20c9fb15a2b1575', 'request_id': 'req-80676137-8f35-45ea-8153-3f028c595ce8'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:39.417907 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cd6b8d-8345-460c-bcba-c054b223a789 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:39.418725 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-80676137-8f35-45ea-8153-3f028c595ce8 admin admin] The token cache is being invalidate because project 5fc6461a081c479fa20c9fb15a2b1575 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81617) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:50:39.419015 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-80676137-8f35-45ea-8153-3f028c595ce8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:39.419668 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-80676137-8f35-45ea-8153-3f028c595ce8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:39.421409 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1382/2753] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:50:39 2022] DELETE /identity/v3/projects/5fc6461a081c479fa20c9fb15a2b1575 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:50:39.426147 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd64ff2a-ae45-4061-9283-3d196c11cb5f None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:39.426551 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd64ff2a-ae45-4061-9283-3d196c11cb5f None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:39.426831 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd64ff2a-ae45-4061-9283-3d196c11cb5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:39.446301 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-cd64ff2a-ae45-4061-9283-3d196c11cb5f None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:50:39.447552 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-cd64ff2a-ae45-4061-9283-3d196c11cb5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:39.470295 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa5e48f-789a-4069-9b30-e13aed1d2764 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:39.471024 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa5e48f-789a-4069-9b30-e13aed1d2764 service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:39.471330 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa5e48f-789a-4069-9b30-e13aed1d2764 service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:39.471609 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa5e48f-789a-4069-9b30-e13aed1d2764 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:39.473344 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-4aa5e48f-789a-4069-9b30-e13aed1d2764 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:39.506173 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1383/2754] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:50:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:50:39.514696 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff7756-65eb-45ed-8cd2-8feb95f1830d None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:39.514980 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff7756-65eb-45ed-8cd2-8feb95f1830d None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:39.515262 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff7756-65eb-45ed-8cd2-8feb95f1830d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:39.537023 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-c6ff7756-65eb-45ed-8cd2-8feb95f1830d None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:50:39.538233 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-c6ff7756-65eb-45ed-8cd2-8feb95f1830d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:39.544562 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa5e48f-789a-4069-9b30-e13aed1d2764 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:39.546008 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa5e48f-789a-4069-9b30-e13aed1d2764 service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:39.549150 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1384/2755] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:50:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:50:39.553204 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ff7756-65eb-45ed-8cd2-8feb95f1830d None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:39.557154 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1372/2756] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:50:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 16:50:39.564192 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ff7756-65eb-45ed-8cd2-8feb95f1830d None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:39.569128 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056b6a3f-989a-42de-a5f8-7ddd0d1824f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:39.569757 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056b6a3f-989a-42de-a5f8-7ddd0d1824f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:39.570035 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056b6a3f-989a-42de-a5f8-7ddd0d1824f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:39.570300 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056b6a3f-989a-42de-a5f8-7ddd0d1824f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:39.571467 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056b6a3f-989a-42de-a5f8-7ddd0d1824f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:50:39.571764 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056b6a3f-989a-42de-a5f8-7ddd0d1824f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:39.573479 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056b6a3f-989a-42de-a5f8-7ddd0d1824f6 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:39.581579 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:50:39.587954 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1385/2757] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:50:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:50:39.592494 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa5e48f-789a-4069-9b30-e13aed1d2764 service neutron] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:39.602285 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa5e48f-789a-4069-9b30-e13aed1d2764 service neutron] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:39.607670 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f4471e-98aa-4ecd-a1b1-513f1ccb7b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:39.608272 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f4471e-98aa-4ecd-a1b1-513f1ccb7b3d admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:39.608550 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f4471e-98aa-4ecd-a1b1-513f1ccb7b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:39.608820 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f4471e-98aa-4ecd-a1b1-513f1ccb7b3d admin admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:39.609467 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f4471e-98aa-4ecd-a1b1-513f1ccb7b3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:39.611388 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f4471e-98aa-4ecd-a1b1-513f1ccb7b3d admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:39.643269 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1373/2758] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:50:39 2022] POST /identity/v3/projects => generated 397 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:50:39.644862 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056b6a3f-989a-42de-a5f8-7ddd0d1824f6 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:39.656541 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056b6a3f-989a-42de-a5f8-7ddd0d1824f6 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:39.720979 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a968bc-5a36-4b6b-917c-b6c5372f9d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:39.721590 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a968bc-5a36-4b6b-917c-b6c5372f9d03 admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:39.721873 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a968bc-5a36-4b6b-917c-b6c5372f9d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:39.722146 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a968bc-5a36-4b6b-917c-b6c5372f9d03 admin admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:39.722825 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a968bc-5a36-4b6b-917c-b6c5372f9d03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:39.724780 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a968bc-5a36-4b6b-917c-b6c5372f9d03 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:39.758913 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1386/2759] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:50:39 2022] POST /identity/v3/users => generated 299 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:50:39.759078 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f4471e-98aa-4ecd-a1b1-513f1ccb7b3d admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:39.768089 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f4471e-98aa-4ecd-a1b1-513f1ccb7b3d admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:39.772882 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c99606-c8c9-4627-b02b-abb96043309b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:39.773492 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c99606-c8c9-4627-b02b-abb96043309b admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:39.774717 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c99606-c8c9-4627-b02b-abb96043309b admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:39.774717 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c99606-c8c9-4627-b02b-abb96043309b admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:39.775351 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c99606-c8c9-4627-b02b-abb96043309b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:39.776845 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c99606-c8c9-4627-b02b-abb96043309b admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:39.791371 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1374/2760] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:50:39 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:50:39.795775 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a968bc-5a36-4b6b-917c-b6c5372f9d03 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:39.808360 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a968bc-5a36-4b6b-917c-b6c5372f9d03 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:39.813263 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2094b4-deba-45a3-9142-2684d084ef04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:39.813892 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2094b4-deba-45a3-9142-2684d084ef04 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:39.814170 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2094b4-deba-45a3-9142-2684d084ef04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:39.814482 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2094b4-deba-45a3-9142-2684d084ef04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:39.815160 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2094b4-deba-45a3-9142-2684d084ef04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:39.816876 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2094b4-deba-45a3-9142-2684d084ef04 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:39.830420 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1387/2761] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:50:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:50:39.832927 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c99606-c8c9-4627-b02b-abb96043309b admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:39.843658 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c99606-c8c9-4627-b02b-abb96043309b admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:39.848679 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2f5e95-728e-42ec-9f81-bf6f1f54469c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:39.849335 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f5e95-728e-42ec-9f81-bf6f1f54469c admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:39.849617 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f5e95-728e-42ec-9f81-bf6f1f54469c admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:39.849896 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f5e95-728e-42ec-9f81-bf6f1f54469c admin admin] PATH_INFO: `/v3/projects/b68ecc9780a144b58944de2555c61cdf/users/77e9228b80ea4933aa22d50dc9e53fc9/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:39.865065 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2f5e95-728e-42ec-9f81-bf6f1f54469c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b68ecc9780a144b58944de2555c61cdf, user_id=77e9228b80ea4933aa22d50dc9e53fc9, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:39.868177 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2f5e95-728e-42ec-9f81-bf6f1f54469c admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:39.886401 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1375/2762] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:50:39 2022] PUT /identity/v3/projects/b68ecc9780a144b58944de2555c61cdf/users/77e9228b80ea4933aa22d50dc9e53fc9/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:50:39.894088 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f79615-4905-4ad5-a3b0-27dcb18500aa None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:39.894395 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f79615-4905-4ad5-a3b0-27dcb18500aa None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:39.894684 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f79615-4905-4ad5-a3b0-27dcb18500aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:39.925426 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-89f79615-4905-4ad5-a3b0-27dcb18500aa None None] MFA Rules not processed for user `77e9228b80ea4933aa22d50dc9e53fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:50:39.926700 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-89f79615-4905-4ad5-a3b0-27dcb18500aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:39.928201 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2f5e95-728e-42ec-9f81-bf6f1f54469c admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:39.929900 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-ab2f5e95-728e-42ec-9f81-bf6f1f54469c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:39.982192 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:50:40.004543 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2f5e95-728e-42ec-9f81-bf6f1f54469c admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:40.009668 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1388/2763] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:50:39 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:50:40.010243 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a55728-9af0-4ca0-aab5-1fac40ec7cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:40.010920 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a55728-9af0-4ca0-aab5-1fac40ec7cb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:40.011226 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a55728-9af0-4ca0-aab5-1fac40ec7cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:40.012039 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a55728-9af0-4ca0-aab5-1fac40ec7cb1 admin admin] PATH_INFO: `/v3/projects/97472977854e45289f84d6732643d0f8` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:40.012672 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a55728-9af0-4ca0-aab5-1fac40ec7cb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97472977854e45289f84d6732643d0f8)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:40.014783 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a55728-9af0-4ca0-aab5-1fac40ec7cb1 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:40.020934 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f79615-4905-4ad5-a3b0-27dcb18500aa None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:40.022897 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-89f79615-4905-4ad5-a3b0-27dcb18500aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:40.074230 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-87a55728-9af0-4ca0-aab5-1fac40ec7cb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97472977854e45289f84d6732643d0f8', 'request_id': 'req-87a55728-9af0-4ca0-aab5-1fac40ec7cb1'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:40.081640 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-87a55728-9af0-4ca0-aab5-1fac40ec7cb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97472977854e45289f84d6732643d0f8', 'request_id': 'req-87a55728-9af0-4ca0-aab5-1fac40ec7cb1'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:40.093043 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-87a55728-9af0-4ca0-aab5-1fac40ec7cb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97472977854e45289f84d6732643d0f8', 'request_id': 'req-87a55728-9af0-4ca0-aab5-1fac40ec7cb1'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:40.098941 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f79615-4905-4ad5-a3b0-27dcb18500aa None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:40.121911 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-87a55728-9af0-4ca0-aab5-1fac40ec7cb1 admin admin] The token cache is being invalidate because project 97472977854e45289f84d6732643d0f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81618) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:50:40.122307 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-87a55728-9af0-4ca0-aab5-1fac40ec7cb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:40.124456 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-87a55728-9af0-4ca0-aab5-1fac40ec7cb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:40.124726 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:50:40.126194 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1376/2764] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:50:39 2022] DELETE /identity/v3/projects/97472977854e45289f84d6732643d0f8 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 16:50:40.131565 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d9b6f2-92d5-4afc-9ff0-62113013a16d None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:40.132670 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d9b6f2-92d5-4afc-9ff0-62113013a16d None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:40.132670 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d9b6f2-92d5-4afc-9ff0-62113013a16d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:40.165781 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-d5d9b6f2-92d5-4afc-9ff0-62113013a16d None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:50:40.167802 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-d5d9b6f2-92d5-4afc-9ff0-62113013a16d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:40.176159 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523be6fd-6d21-4708-91bd-ddf5eb6af9ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:40.176840 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523be6fd-6d21-4708-91bd-ddf5eb6af9ed service cinder] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:40.177219 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523be6fd-6d21-4708-91bd-ddf5eb6af9ed service cinder] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:40.177588 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523be6fd-6d21-4708-91bd-ddf5eb6af9ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:40.179557 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-523be6fd-6d21-4708-91bd-ddf5eb6af9ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:40.241476 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1377/2765] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:50:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:50:40.249269 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c62cf10-6f37-41ba-920e-51b74fdb9d5c None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:40.249559 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c62cf10-6f37-41ba-920e-51b74fdb9d5c None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:40.249839 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c62cf10-6f37-41ba-920e-51b74fdb9d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:40.276102 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523be6fd-6d21-4708-91bd-ddf5eb6af9ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:40.276830 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-6c62cf10-6f37-41ba-920e-51b74fdb9d5c None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:50:40.278091 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523be6fd-6d21-4708-91bd-ddf5eb6af9ed service cinder] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:40.279902 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-6c62cf10-6f37-41ba-920e-51b74fdb9d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:40.288760 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1378/2766] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:50:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:50:40.290583 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1389/2767] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:50:40 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 16:50:40.293930 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c62cf10-6f37-41ba-920e-51b74fdb9d5c None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:40.306177 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c62cf10-6f37-41ba-920e-51b74fdb9d5c None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:40.315174 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd8b365-2151-4901-a615-9db3526a8ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:40.315803 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd8b365-2151-4901-a615-9db3526a8ee8 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:40.316086 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd8b365-2151-4901-a615-9db3526a8ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:40.316407 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd8b365-2151-4901-a615-9db3526a8ee8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:40.317021 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd8b365-2151-4901-a615-9db3526a8ee8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:50:40.317326 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd8b365-2151-4901-a615-9db3526a8ee8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:40.324019 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd8b365-2151-4901-a615-9db3526a8ee8 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:40.341938 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1379/2768] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:50:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:50:40.343067 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523be6fd-6d21-4708-91bd-ddf5eb6af9ed service cinder] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:40.356366 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523be6fd-6d21-4708-91bd-ddf5eb6af9ed service cinder] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:40.361641 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ad85d-3d38-4970-b1af-ef64d39431ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:40.362245 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ad85d-3d38-4970-b1af-ef64d39431ff admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:40.362558 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ad85d-3d38-4970-b1af-ef64d39431ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:40.362831 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ad85d-3d38-4970-b1af-ef64d39431ff admin admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:40.363467 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ad85d-3d38-4970-b1af-ef64d39431ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:40.365371 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ad85d-3d38-4970-b1af-ef64d39431ff admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:40.401689 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1390/2769] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:50:40 2022] POST /identity/v3/projects => generated 365 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:50:40.409368 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd8b365-2151-4901-a615-9db3526a8ee8 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:40.425155 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd8b365-2151-4901-a615-9db3526a8ee8 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:40.488055 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:50:40.499496 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73c0455-affc-47af-afee-bd99384f5803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:40.500673 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c0455-affc-47af-afee-bd99384f5803 admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:40.501007 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c0455-affc-47af-afee-bd99384f5803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:40.501299 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c0455-affc-47af-afee-bd99384f5803 admin admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:40.502337 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73c0455-affc-47af-afee-bd99384f5803 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:40.505737 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73c0455-affc-47af-afee-bd99384f5803 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:40.539789 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1380/2770] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:50:40 2022] POST /identity/v3/users => generated 283 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:50:40.543076 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ad85d-3d38-4970-b1af-ef64d39431ff admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:40.562965 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ad85d-3d38-4970-b1af-ef64d39431ff admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:40.569775 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07ebaa2-aff6-451a-ad96-aa1252be963e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:40.571288 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07ebaa2-aff6-451a-ad96-aa1252be963e admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:40.571636 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07ebaa2-aff6-451a-ad96-aa1252be963e admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:40.571972 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07ebaa2-aff6-451a-ad96-aa1252be963e admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:40.572691 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07ebaa2-aff6-451a-ad96-aa1252be963e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:40.578245 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07ebaa2-aff6-451a-ad96-aa1252be963e admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:40.603804 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1391/2771] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:50:40 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 16:50:40.605582 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73c0455-affc-47af-afee-bd99384f5803 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:40.623270 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73c0455-affc-47af-afee-bd99384f5803 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:40.631809 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0079560e-ee19-4044-b152-8b00a156f8c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:40.632601 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0079560e-ee19-4044-b152-8b00a156f8c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:40.632924 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0079560e-ee19-4044-b152-8b00a156f8c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:40.633763 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0079560e-ee19-4044-b152-8b00a156f8c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:40.634444 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0079560e-ee19-4044-b152-8b00a156f8c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:40.636385 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0079560e-ee19-4044-b152-8b00a156f8c4 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:40.656996 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1381/2772] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:50:40 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:50:40.661770 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07ebaa2-aff6-451a-ad96-aa1252be963e admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:40.686055 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07ebaa2-aff6-451a-ad96-aa1252be963e admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:40.699241 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4bf7ba-a2b0-44e9-96e9-924ba4164816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:40.699920 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4bf7ba-a2b0-44e9-96e9-924ba4164816 admin admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:40.700231 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4bf7ba-a2b0-44e9-96e9-924ba4164816 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:40.700513 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4bf7ba-a2b0-44e9-96e9-924ba4164816 admin admin] PATH_INFO: `/v3/projects/232013e158a74a14be6868895aa879ef/users/26cc26277d9a4d0589c13356cbf158dd/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:40.722652 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4bf7ba-a2b0-44e9-96e9-924ba4164816 admin admin] RBAC: Authorizing `identity:create_grant(project_id=232013e158a74a14be6868895aa879ef, user_id=26cc26277d9a4d0589c13356cbf158dd, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:40.725912 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4bf7ba-a2b0-44e9-96e9-924ba4164816 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:40.752721 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332fbd59-947a-45dc-9d21-8dba090fda63 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:40.753030 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332fbd59-947a-45dc-9d21-8dba090fda63 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:40.753306 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332fbd59-947a-45dc-9d21-8dba090fda63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:40.756363 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1392/2773] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:50:40 2022] PUT /identity/v3/projects/232013e158a74a14be6868895aa879ef/users/26cc26277d9a4d0589c13356cbf158dd/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:50:40.807223 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-332fbd59-947a-45dc-9d21-8dba090fda63 None None] MFA Rules not processed for user `26cc26277d9a4d0589c13356cbf158dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:50:40.810205 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-332fbd59-947a-45dc-9d21-8dba090fda63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:40.938562 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1382/2774] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:50:40 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:50:40.951212 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4bf7ba-a2b0-44e9-96e9-924ba4164816 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:40.953519 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-9e4bf7ba-a2b0-44e9-96e9-924ba4164816 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:41.025267 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4bf7ba-a2b0-44e9-96e9-924ba4164816 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:41.028875 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24a6951-4b6d-4a68-8a3e-f95c03e5f4d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:41.029488 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a6951-4b6d-4a68-8a3e-f95c03e5f4d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:41.029761 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a6951-4b6d-4a68-8a3e-f95c03e5f4d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:41.030031 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a6951-4b6d-4a68-8a3e-f95c03e5f4d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:41.038343 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24a6951-4b6d-4a68-8a3e-f95c03e5f4d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:41.040324 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24a6951-4b6d-4a68-8a3e-f95c03e5f4d8 service cinder] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:41.049314 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1393/2775] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:50:40 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:50:42.146655 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a036045-b081-4ee8-a995-3a39c18374e0 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:42.147176 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a036045-b081-4ee8-a995-3a39c18374e0 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:42.147289 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a036045-b081-4ee8-a995-3a39c18374e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:42.169258 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-8a036045-b081-4ee8-a995-3a39c18374e0 None None] MFA Rules not processed for user `7222bd3b313b47e5a4fa0ebc9ff0cdd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:50:42.170596 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-8a036045-b081-4ee8-a995-3a39c18374e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:42.237728 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1383/2776] 10.222.0.57 () {58 vars in 1009 bytes} [Mon Sep 19 16:50:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:50:42.260980 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24a6951-4b6d-4a68-8a3e-f95c03e5f4d8 service cinder] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:42.262413 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-e24a6951-4b6d-4a68-8a3e-f95c03e5f4d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:42.292547 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:50:42.340171 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24a6951-4b6d-4a68-8a3e-f95c03e5f4d8 service cinder] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:42.345234 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d545bd-1f31-4ca8-80a6-b063e1c69dcd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '884a0524aca744b99c7496c11c3291ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:42.345842 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d545bd-1f31-4ca8-80a6-b063e1c69dcd service swift] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:42.346112 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d545bd-1f31-4ca8-80a6-b063e1c69dcd service swift] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:42.346391 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d545bd-1f31-4ca8-80a6-b063e1c69dcd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:42.356098 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d545bd-1f31-4ca8-80a6-b063e1c69dcd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:42.357732 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d545bd-1f31-4ca8-80a6-b063e1c69dcd service swift] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:42.366632 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1394/2777] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Sep 19 16:50:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 19 16:50:42.948242 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81051e8-67dd-4d5a-ae6c-865d6c360a6f None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:42.948635 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81051e8-67dd-4d5a-ae6c-865d6c360a6f None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:42.948999 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81051e8-67dd-4d5a-ae6c-865d6c360a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:42.970185 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-d81051e8-67dd-4d5a-ae6c-865d6c360a6f None None] MFA Rules not processed for user `7222bd3b313b47e5a4fa0ebc9ff0cdd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:50:42.971542 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-d81051e8-67dd-4d5a-ae6c-865d6c360a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:42.979556 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1384/2778] 10.222.0.57 () {58 vars in 967 bytes} [Mon Sep 19 16:50:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:50:42.993517 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d545bd-1f31-4ca8-80a6-b063e1c69dcd service swift] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:43.005724 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d545bd-1f31-4ca8-80a6-b063e1c69dcd service swift] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:43.011881 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11a260f-9d06-4c85-87e4-ba6b2d754916 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '884a0524aca744b99c7496c11c3291ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:43.012531 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11a260f-9d06-4c85-87e4-ba6b2d754916 service swift] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:43.012811 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11a260f-9d06-4c85-87e4-ba6b2d754916 service swift] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:43.013088 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11a260f-9d06-4c85-87e4-ba6b2d754916 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:43.022177 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11a260f-9d06-4c85-87e4-ba6b2d754916 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:43.024036 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11a260f-9d06-4c85-87e4-ba6b2d754916 service swift] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:43.033945 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1395/2779] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Sep 19 16:50:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 16:50:43.270408 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c875c89b-00ef-40ca-a5f5-d1a9c2842045 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:43.270942 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c875c89b-00ef-40ca-a5f5-d1a9c2842045 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:43.270942 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c875c89b-00ef-40ca-a5f5-d1a9c2842045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:43.291012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-c875c89b-00ef-40ca-a5f5-d1a9c2842045 None None] MFA Rules not processed for user `7222bd3b313b47e5a4fa0ebc9ff0cdd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:50:43.292259 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-c875c89b-00ef-40ca-a5f5-d1a9c2842045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:43.299566 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1385/2780] 10.222.0.57 () {58 vars in 1009 bytes} [Mon Sep 19 16:50:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:50:43.312389 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11a260f-9d06-4c85-87e4-ba6b2d754916 service swift] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:43.322878 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11a260f-9d06-4c85-87e4-ba6b2d754916 service swift] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:43.328143 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e473dd54-21c0-47cb-b9ed-b16a6735fe75 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '884a0524aca744b99c7496c11c3291ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:43.328737 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473dd54-21c0-47cb-b9ed-b16a6735fe75 service swift] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:43.329044 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473dd54-21c0-47cb-b9ed-b16a6735fe75 service swift] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:43.329405 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473dd54-21c0-47cb-b9ed-b16a6735fe75 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:43.337810 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e473dd54-21c0-47cb-b9ed-b16a6735fe75 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:43.339421 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e473dd54-21c0-47cb-b9ed-b16a6735fe75 service swift] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:43.347578 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1396/2781] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Sep 19 16:50:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 16:50:43.683659 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4713bd1-0662-4b49-b9c9-13871fb9f838 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:43.683944 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4713bd1-0662-4b49-b9c9-13871fb9f838 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:43.684217 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4713bd1-0662-4b49-b9c9-13871fb9f838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:43.705027 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-b4713bd1-0662-4b49-b9c9-13871fb9f838 None None] MFA Rules not processed for user `7222bd3b313b47e5a4fa0ebc9ff0cdd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:50:43.706260 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-b4713bd1-0662-4b49-b9c9-13871fb9f838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:43.718099 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1386/2782] 10.222.0.57 () {58 vars in 967 bytes} [Mon Sep 19 16:50:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:50:43.732604 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e473dd54-21c0-47cb-b9ed-b16a6735fe75 service swift] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:43.745717 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e473dd54-21c0-47cb-b9ed-b16a6735fe75 service swift] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:43.752126 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4306f5-72d9-4217-9de4-76cebbfb8d09 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '884a0524aca744b99c7496c11c3291ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:43.753157 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4306f5-72d9-4217-9de4-76cebbfb8d09 service swift] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:43.753436 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4306f5-72d9-4217-9de4-76cebbfb8d09 service swift] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:43.753712 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4306f5-72d9-4217-9de4-76cebbfb8d09 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:43.762750 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4306f5-72d9-4217-9de4-76cebbfb8d09 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:43.764335 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4306f5-72d9-4217-9de4-76cebbfb8d09 service swift] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:43.772685 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1397/2783] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Sep 19 16:50:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 16:50:45.888787 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4713bd1-0662-4b49-b9c9-13871fb9f838 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:45.890095 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-b4713bd1-0662-4b49-b9c9-13871fb9f838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:45.938919 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:50:45.955960 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4713bd1-0662-4b49-b9c9-13871fb9f838 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:45.960492 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ac857f-d087-4583-aa4e-91fb27fb11ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:45.961119 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac857f-d087-4583-aa4e-91fb27fb11ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:45.961881 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac857f-d087-4583-aa4e-91fb27fb11ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:45.962154 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac857f-d087-4583-aa4e-91fb27fb11ac admin admin] PATH_INFO: `/v3/users/c4bfbedbe6cd494984e181a29c36ed8c` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:45.963322 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ac857f-d087-4583-aa4e-91fb27fb11ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4bfbedbe6cd494984e181a29c36ed8c)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:45.965249 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ac857f-d087-4583-aa4e-91fb27fb11ac admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:46.009181 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-35ac857f-d087-4583-aa4e-91fb27fb11ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4bfbedbe6cd494984e181a29c36ed8c'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:46.015827 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-35ac857f-d087-4583-aa4e-91fb27fb11ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4bfbedbe6cd494984e181a29c36ed8c'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:46.027845 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-35ac857f-d087-4583-aa4e-91fb27fb11ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4bfbedbe6cd494984e181a29c36ed8c'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:46.035020 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-35ac857f-d087-4583-aa4e-91fb27fb11ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4bfbedbe6cd494984e181a29c36ed8c'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:46.035633 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-35ac857f-d087-4583-aa4e-91fb27fb11ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4bfbedbe6cd494984e181a29c36ed8c'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:46.042009 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1387/2784] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:50:45 2022] DELETE /identity/v3/users/c4bfbedbe6cd494984e181a29c36ed8c => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:50:46.046758 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad1c885-5242-4c0a-b568-e58871257cd7 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:46.047256 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad1c885-5242-4c0a-b568-e58871257cd7 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:46.047383 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad1c885-5242-4c0a-b568-e58871257cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:46.066407 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-6ad1c885-5242-4c0a-b568-e58871257cd7 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:50:46.067602 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-6ad1c885-5242-4c0a-b568-e58871257cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:46.130446 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1398/2785] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:50:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:50:46.141256 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ac857f-d087-4583-aa4e-91fb27fb11ac admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:46.142686 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-35ac857f-d087-4583-aa4e-91fb27fb11ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:46.202586 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ac857f-d087-4583-aa4e-91fb27fb11ac admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:46.211040 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c534e08f-146b-4989-b6e2-f55dba5948d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:46.211040 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c534e08f-146b-4989-b6e2-f55dba5948d2 service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:46.211040 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c534e08f-146b-4989-b6e2-f55dba5948d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:46.211040 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c534e08f-146b-4989-b6e2-f55dba5948d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:46.220279 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c534e08f-146b-4989-b6e2-f55dba5948d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:46.221720 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c534e08f-146b-4989-b6e2-f55dba5948d2 service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:46.231973 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1388/2786] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:50:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:50:46.424824 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad1c885-5242-4c0a-b568-e58871257cd7 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:46.426559 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-6ad1c885-5242-4c0a-b568-e58871257cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:46.441988 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad1c885-5242-4c0a-b568-e58871257cd7 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:46.450932 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7220bdad-3407-44f3-adf3-7a96eeb39a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:46.451708 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7220bdad-3407-44f3-adf3-7a96eeb39a96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:46.451991 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7220bdad-3407-44f3-adf3-7a96eeb39a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:46.452266 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7220bdad-3407-44f3-adf3-7a96eeb39a96 admin admin] PATH_INFO: `/v3/projects/9026e39904fa45eb9984dc3d687e7a08` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:46.453616 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7220bdad-3407-44f3-adf3-7a96eeb39a96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9026e39904fa45eb9984dc3d687e7a08)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:46.456191 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7220bdad-3407-44f3-adf3-7a96eeb39a96 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:46.509181 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-7220bdad-3407-44f3-adf3-7a96eeb39a96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9026e39904fa45eb9984dc3d687e7a08', 'request_id': 'req-7220bdad-3407-44f3-adf3-7a96eeb39a96'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:46.516565 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-7220bdad-3407-44f3-adf3-7a96eeb39a96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9026e39904fa45eb9984dc3d687e7a08', 'request_id': 'req-7220bdad-3407-44f3-adf3-7a96eeb39a96'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:46.522951 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-7220bdad-3407-44f3-adf3-7a96eeb39a96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9026e39904fa45eb9984dc3d687e7a08', 'request_id': 'req-7220bdad-3407-44f3-adf3-7a96eeb39a96'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:46.527805 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:50:46.538959 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-7220bdad-3407-44f3-adf3-7a96eeb39a96 admin admin] The token cache is being invalidate because project 9026e39904fa45eb9984dc3d687e7a08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81617) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:50:46.539249 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-7220bdad-3407-44f3-adf3-7a96eeb39a96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:46.539921 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-7220bdad-3407-44f3-adf3-7a96eeb39a96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:46.541415 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1399/2787] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:50:46 2022] DELETE /identity/v3/projects/9026e39904fa45eb9984dc3d687e7a08 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:50:48.644892 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c534e08f-146b-4989-b6e2-f55dba5948d2 service neutron] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:48.646304 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-c534e08f-146b-4989-b6e2-f55dba5948d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:48.659298 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c534e08f-146b-4989-b6e2-f55dba5948d2 service neutron] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:48.734230 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfb4db5-cb5d-4b8d-a9d1-8969d227189a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:48.735025 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb4db5-cb5d-4b8d-a9d1-8969d227189a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:48.735375 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb4db5-cb5d-4b8d-a9d1-8969d227189a admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:48.735719 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb4db5-cb5d-4b8d-a9d1-8969d227189a admin admin] PATH_INFO: `/v3/users/77e9228b80ea4933aa22d50dc9e53fc9` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:48.737208 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfb4db5-cb5d-4b8d-a9d1-8969d227189a admin admin] RBAC: Authorizing `identity:delete_user(user_id=77e9228b80ea4933aa22d50dc9e53fc9)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:48.739575 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfb4db5-cb5d-4b8d-a9d1-8969d227189a admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:48.759953 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:50:48.798637 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-4bfb4db5-cb5d-4b8d-a9d1-8969d227189a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77e9228b80ea4933aa22d50dc9e53fc9'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:48.808116 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-4bfb4db5-cb5d-4b8d-a9d1-8969d227189a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77e9228b80ea4933aa22d50dc9e53fc9'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:48.820938 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-4bfb4db5-cb5d-4b8d-a9d1-8969d227189a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77e9228b80ea4933aa22d50dc9e53fc9'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:48.828121 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-4bfb4db5-cb5d-4b8d-a9d1-8969d227189a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77e9228b80ea4933aa22d50dc9e53fc9'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:48.829021 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-4bfb4db5-cb5d-4b8d-a9d1-8969d227189a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77e9228b80ea4933aa22d50dc9e53fc9'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:48.835253 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1389/2788] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:50:48 2022] DELETE /identity/v3/users/77e9228b80ea4933aa22d50dc9e53fc9 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:50:48.839828 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee2634c-f929-44a7-96c9-f1b847532265 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:48.840313 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee2634c-f929-44a7-96c9-f1b847532265 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:48.840444 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee2634c-f929-44a7-96c9-f1b847532265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:48.859675 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-5ee2634c-f929-44a7-96c9-f1b847532265 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:50:48.860893 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-5ee2634c-f929-44a7-96c9-f1b847532265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:48.919412 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1400/2789] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:50:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:50:48.929208 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfb4db5-cb5d-4b8d-a9d1-8969d227189a admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:48.930669 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-4bfb4db5-cb5d-4b8d-a9d1-8969d227189a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:48.990052 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfb4db5-cb5d-4b8d-a9d1-8969d227189a admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:48.993846 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6982e86f-b34a-4e4f-8fdb-17c2d13e353d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:48.994467 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6982e86f-b34a-4e4f-8fdb-17c2d13e353d service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:48.994745 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6982e86f-b34a-4e4f-8fdb-17c2d13e353d service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:48.995012 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6982e86f-b34a-4e4f-8fdb-17c2d13e353d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:49.006645 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6982e86f-b34a-4e4f-8fdb-17c2d13e353d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:49.008062 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6982e86f-b34a-4e4f-8fdb-17c2d13e353d service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:49.017954 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1390/2790] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:50:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:50:49.295992 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee2634c-f929-44a7-96c9-f1b847532265 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:49.297422 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-5ee2634c-f929-44a7-96c9-f1b847532265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:49.310848 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee2634c-f929-44a7-96c9-f1b847532265 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:49.315630 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf75d398-6fda-4e9a-b411-75deabdde9be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:49.316274 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75d398-6fda-4e9a-b411-75deabdde9be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:49.316555 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75d398-6fda-4e9a-b411-75deabdde9be admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:49.316833 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75d398-6fda-4e9a-b411-75deabdde9be admin admin] PATH_INFO: `/v3/projects/b68ecc9780a144b58944de2555c61cdf` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:49.317973 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf75d398-6fda-4e9a-b411-75deabdde9be admin admin] RBAC: Authorizing `identity:delete_project(project_id=b68ecc9780a144b58944de2555c61cdf)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:49.319925 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf75d398-6fda-4e9a-b411-75deabdde9be admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:49.368512 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-bf75d398-6fda-4e9a-b411-75deabdde9be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b68ecc9780a144b58944de2555c61cdf', 'request_id': 'req-bf75d398-6fda-4e9a-b411-75deabdde9be'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:49.374678 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-bf75d398-6fda-4e9a-b411-75deabdde9be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b68ecc9780a144b58944de2555c61cdf', 'request_id': 'req-bf75d398-6fda-4e9a-b411-75deabdde9be'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:49.380698 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-bf75d398-6fda-4e9a-b411-75deabdde9be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b68ecc9780a144b58944de2555c61cdf', 'request_id': 'req-bf75d398-6fda-4e9a-b411-75deabdde9be'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:49.393372 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-bf75d398-6fda-4e9a-b411-75deabdde9be admin admin] The token cache is being invalidate because project b68ecc9780a144b58944de2555c61cdf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81617) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:50:49.393680 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-bf75d398-6fda-4e9a-b411-75deabdde9be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:49.394311 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-bf75d398-6fda-4e9a-b411-75deabdde9be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:50:49.395824 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1401/2791] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:50:49 2022] DELETE /identity/v3/projects/b68ecc9780a144b58944de2555c61cdf => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:50:49.401687 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc03de3-b602-4f6c-a39f-be1787675bab None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:49.401996 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc03de3-b602-4f6c-a39f-be1787675bab None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:49.402270 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc03de3-b602-4f6c-a39f-be1787675bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:49.421341 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-1cc03de3-b602-4f6c-a39f-be1787675bab None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:50:49.422579 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-1cc03de3-b602-4f6c-a39f-be1787675bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:49.492536 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1391/2792] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:50:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:50:49.501761 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fbc3cb-6f31-4eeb-9511-9e2396917f3a None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:49.502043 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fbc3cb-6f31-4eeb-9511-9e2396917f3a None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:49.502322 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fbc3cb-6f31-4eeb-9511-9e2396917f3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:49.522541 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-01fbc3cb-6f31-4eeb-9511-9e2396917f3a None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:50:49.523804 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-01fbc3cb-6f31-4eeb-9511-9e2396917f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:49.532828 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1402/2793] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:50:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:50:49.537067 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc03de3-b602-4f6c-a39f-be1787675bab None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:49.546737 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc03de3-b602-4f6c-a39f-be1787675bab None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:49.551512 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9921423-88b8-4215-a1c2-e38aa358789f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:49.552106 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9921423-88b8-4215-a1c2-e38aa358789f admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:49.552388 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9921423-88b8-4215-a1c2-e38aa358789f admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:49.552663 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9921423-88b8-4215-a1c2-e38aa358789f admin admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:49.553307 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9921423-88b8-4215-a1c2-e38aa358789f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:50:49.553613 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9921423-88b8-4215-a1c2-e38aa358789f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:49.555232 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9921423-88b8-4215-a1c2-e38aa358789f admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:49.566882 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1392/2794] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:50:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:50:49.571772 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fbc3cb-6f31-4eeb-9511-9e2396917f3a None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:49.581057 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fbc3cb-6f31-4eeb-9511-9e2396917f3a None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:49.585758 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6c2bd7-720c-4d3a-bd2d-b5f3c33ef6f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:49.586999 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c2bd7-720c-4d3a-bd2d-b5f3c33ef6f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:49.587285 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c2bd7-720c-4d3a-bd2d-b5f3c33ef6f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:49.587559 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c2bd7-720c-4d3a-bd2d-b5f3c33ef6f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:49.588200 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6c2bd7-720c-4d3a-bd2d-b5f3c33ef6f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:49.590088 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6c2bd7-720c-4d3a-bd2d-b5f3c33ef6f9 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:49.620036 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1403/2795] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:50:49 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:50:49.625289 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9921423-88b8-4215-a1c2-e38aa358789f admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:49.637617 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9921423-88b8-4215-a1c2-e38aa358789f admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:49.655333 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:50:49.695478 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e7df07-ea21-4d80-882b-e9435288353f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:49.696691 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e7df07-ea21-4d80-882b-e9435288353f admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:49.696968 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e7df07-ea21-4d80-882b-e9435288353f admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:49.697247 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e7df07-ea21-4d80-882b-e9435288353f admin admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:49.697860 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e7df07-ea21-4d80-882b-e9435288353f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:49.699780 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e7df07-ea21-4d80-882b-e9435288353f admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:49.721257 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1393/2796] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:50:49 2022] POST /identity/v3/users => generated 286 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 16:50:49.725561 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6c2bd7-720c-4d3a-bd2d-b5f3c33ef6f9 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:49.735834 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6c2bd7-720c-4d3a-bd2d-b5f3c33ef6f9 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:49.740454 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0e9d62-4e72-41ad-8418-5260d434c27d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:49.741049 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e9d62-4e72-41ad-8418-5260d434c27d admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:49.741325 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e9d62-4e72-41ad-8418-5260d434c27d admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:49.741608 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e9d62-4e72-41ad-8418-5260d434c27d admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:49.742237 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0e9d62-4e72-41ad-8418-5260d434c27d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:49.744431 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0e9d62-4e72-41ad-8418-5260d434c27d admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:49.754274 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1404/2797] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:50:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:50:49.758792 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e7df07-ea21-4d80-882b-e9435288353f admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:49.769647 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e7df07-ea21-4d80-882b-e9435288353f admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:49.774748 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80ea674-a281-4196-923d-c9119195bba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:49.775368 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ea674-a281-4196-923d-c9119195bba0 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:49.775653 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ea674-a281-4196-923d-c9119195bba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:49.775924 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ea674-a281-4196-923d-c9119195bba0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:49.776709 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80ea674-a281-4196-923d-c9119195bba0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:49.778420 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80ea674-a281-4196-923d-c9119195bba0 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:49.790224 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1394/2798] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:50:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:50:49.794394 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0e9d62-4e72-41ad-8418-5260d434c27d admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:49.804351 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0e9d62-4e72-41ad-8418-5260d434c27d admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:49.810204 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9e4161-8bf7-4103-9ca1-84deb2ad0309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:49.810889 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e4161-8bf7-4103-9ca1-84deb2ad0309 admin admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:49.811169 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e4161-8bf7-4103-9ca1-84deb2ad0309 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:49.811442 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e4161-8bf7-4103-9ca1-84deb2ad0309 admin admin] PATH_INFO: `/v3/projects/b62d50ba5e3d451dab48a4b6cdc50399/users/8a3f7b1c624141feab14c67fa39732a1/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:49.829730 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9e4161-8bf7-4103-9ca1-84deb2ad0309 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b62d50ba5e3d451dab48a4b6cdc50399, user_id=8a3f7b1c624141feab14c67fa39732a1, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:49.832618 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9e4161-8bf7-4103-9ca1-84deb2ad0309 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:49.846451 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1405/2799] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:50:49 2022] PUT /identity/v3/projects/b62d50ba5e3d451dab48a4b6cdc50399/users/8a3f7b1c624141feab14c67fa39732a1/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:50:49.851555 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afe3d6c-d286-40de-aad3-3fe9ecc872d0 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:49.851856 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afe3d6c-d286-40de-aad3-3fe9ecc872d0 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:49.852147 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afe3d6c-d286-40de-aad3-3fe9ecc872d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:49.871808 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-6afe3d6c-d286-40de-aad3-3fe9ecc872d0 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:50:49.873032 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-6afe3d6c-d286-40de-aad3-3fe9ecc872d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:49.934796 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1395/2800] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:50:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:50:49.944495 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9e4161-8bf7-4103-9ca1-84deb2ad0309 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:49.946824 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-ad9e4161-8bf7-4103-9ca1-84deb2ad0309 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:49.967956 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:50:50.012286 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9e4161-8bf7-4103-9ca1-84deb2ad0309 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:50.016316 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fec319-9642-4f57-8891-69f0aff9f42f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:50.016934 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fec319-9642-4f57-8891-69f0aff9f42f service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:50.017215 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fec319-9642-4f57-8891-69f0aff9f42f service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:50.017488 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fec319-9642-4f57-8891-69f0aff9f42f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:50.029377 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fec319-9642-4f57-8891-69f0aff9f42f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:50.030954 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fec319-9642-4f57-8891-69f0aff9f42f service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:50.042648 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1406/2801] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:50:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:50:54.172182 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d3a192-e0f7-4d0f-b460-1a8ecb853ad5 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:54.172669 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d3a192-e0f7-4d0f-b460-1a8ecb853ad5 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:54.172899 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d3a192-e0f7-4d0f-b460-1a8ecb853ad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:54.202051 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-28d3a192-e0f7-4d0f-b460-1a8ecb853ad5 None None] MFA Rules not processed for user `8a3f7b1c624141feab14c67fa39732a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:50:54.203430 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-28d3a192-e0f7-4d0f-b460-1a8ecb853ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:50:54.286660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1396/2802] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:50:54 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:50:54.291717 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fec319-9642-4f57-8891-69f0aff9f42f service neutron] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:54.301777 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fec319-9642-4f57-8891-69f0aff9f42f service neutron] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:54.307642 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a159349-9ecb-4a74-80e6-c0818ddaec4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:54.308081 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a159349-9ecb-4a74-80e6-c0818ddaec4f admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:54.308361 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a159349-9ecb-4a74-80e6-c0818ddaec4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:54.308625 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a159349-9ecb-4a74-80e6-c0818ddaec4f admin admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:54.309598 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a159349-9ecb-4a74-80e6-c0818ddaec4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:54.312379 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a159349-9ecb-4a74-80e6-c0818ddaec4f admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:54.344249 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1407/2803] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:50:54 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:50:54.348704 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d3a192-e0f7-4d0f-b460-1a8ecb853ad5 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:54.361123 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d3a192-e0f7-4d0f-b460-1a8ecb853ad5 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:54.390194 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:50:54.434494 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93ef7d3-e26d-4bcf-b821-cc3ebec1f723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:54.435113 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ef7d3-e26d-4bcf-b821-cc3ebec1f723 admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:54.435434 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ef7d3-e26d-4bcf-b821-cc3ebec1f723 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:54.435668 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ef7d3-e26d-4bcf-b821-cc3ebec1f723 admin admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:54.436286 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93ef7d3-e26d-4bcf-b821-cc3ebec1f723 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:54.438346 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93ef7d3-e26d-4bcf-b821-cc3ebec1f723 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:54.462342 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1397/2804] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:50:54 2022] POST /identity/v3/users => generated 293 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:50:54.466243 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a159349-9ecb-4a74-80e6-c0818ddaec4f admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:54.481879 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a159349-9ecb-4a74-80e6-c0818ddaec4f admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:54.486844 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c24654-a397-41b6-9e89-11bc86d7373a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:54.487583 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c24654-a397-41b6-9e89-11bc86d7373a admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:54.487911 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c24654-a397-41b6-9e89-11bc86d7373a admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:54.488270 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c24654-a397-41b6-9e89-11bc86d7373a admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:54.488989 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c24654-a397-41b6-9e89-11bc86d7373a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:54.490803 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c24654-a397-41b6-9e89-11bc86d7373a admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:54.502640 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1408/2805] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:50:54 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 16:50:54.507069 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93ef7d3-e26d-4bcf-b821-cc3ebec1f723 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:54.520014 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93ef7d3-e26d-4bcf-b821-cc3ebec1f723 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:54.525666 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec43333-b9f7-4be8-8b70-26ae171fa308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:54.526520 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec43333-b9f7-4be8-8b70-26ae171fa308 admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:54.526866 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec43333-b9f7-4be8-8b70-26ae171fa308 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:54.527205 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec43333-b9f7-4be8-8b70-26ae171fa308 admin admin] PATH_INFO: `/v3/projects/27c3d04088f9422492d1549c6663bde4/users/9a0f1c6dfd8243b0958a5c7517af5438/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:54.545908 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec43333-b9f7-4be8-8b70-26ae171fa308 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27c3d04088f9422492d1549c6663bde4, user_id=9a0f1c6dfd8243b0958a5c7517af5438, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:54.547972 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec43333-b9f7-4be8-8b70-26ae171fa308 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:54.565563 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1398/2806] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:50:54 2022] PUT /identity/v3/projects/27c3d04088f9422492d1549c6663bde4/users/9a0f1c6dfd8243b0958a5c7517af5438/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:50:54.569560 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c24654-a397-41b6-9e89-11bc86d7373a admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:54.581143 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c24654-a397-41b6-9e89-11bc86d7373a admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:54.674026 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcd18f1-2c3c-498a-82e1-30144284331b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:54.674755 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd18f1-2c3c-498a-82e1-30144284331b admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:54.675123 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd18f1-2c3c-498a-82e1-30144284331b admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:54.675477 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd18f1-2c3c-498a-82e1-30144284331b admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:54.676225 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcd18f1-2c3c-498a-82e1-30144284331b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:54.679580 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcd18f1-2c3c-498a-82e1-30144284331b admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:54.691228 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1409/2807] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:50:54 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 16:50:54.696009 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec43333-b9f7-4be8-8b70-26ae171fa308 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:50:54.707055 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec43333-b9f7-4be8-8b70-26ae171fa308 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:50:54.714916 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e5266b-2c99-4a62-8e09-bbd37400b3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:50:54.715601 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5266b-2c99-4a62-8e09-bbd37400b3a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:50:54.715883 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5266b-2c99-4a62-8e09-bbd37400b3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:50:54.716155 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5266b-2c99-4a62-8e09-bbd37400b3a4 admin admin] PATH_INFO: `/v3/projects/27c3d04088f9422492d1549c6663bde4/users/9a0f1c6dfd8243b0958a5c7517af5438/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:50:54.718474 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e5266b-2c99-4a62-8e09-bbd37400b3a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27c3d04088f9422492d1549c6663bde4, user_id=9a0f1c6dfd8243b0958a5c7517af5438, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:50:54.721121 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e5266b-2c99-4a62-8e09-bbd37400b3a4 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:50:54.738421 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1399/2808] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:50:54 2022] PUT /identity/v3/projects/27c3d04088f9422492d1549c6663bde4/users/9a0f1c6dfd8243b0958a5c7517af5438/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 16:51:01.034340 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822a9f3-b670-4642-b91a-dd8096fb194f None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:51:01.034340 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822a9f3-b670-4642-b91a-dd8096fb194f None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:51:01.034340 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822a9f3-b670-4642-b91a-dd8096fb194f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:51:01.076954 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-9822a9f3-b670-4642-b91a-dd8096fb194f None None] MFA Rules not processed for user `9a0f1c6dfd8243b0958a5c7517af5438`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:51:01.078826 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-9822a9f3-b670-4642-b91a-dd8096fb194f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:51:01.176774 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1410/2809] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:51:01 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:51:01.191630 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e5266b-2c99-4a62-8e09-bbd37400b3a4 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:51:01.193225 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-39e5266b-2c99-4a62-8e09-bbd37400b3a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:51:01.281180 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e5266b-2c99-4a62-8e09-bbd37400b3a4 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:51:01.287531 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312f429d-3406-4680-88bb-c69160b37e84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:51:01.288264 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f429d-3406-4680-88bb-c69160b37e84 service nova] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:51:01.288607 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f429d-3406-4680-88bb-c69160b37e84 service nova] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:51:01.288844 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f429d-3406-4680-88bb-c69160b37e84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:51:01.352716 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312f429d-3406-4680-88bb-c69160b37e84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:51:01.354317 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312f429d-3406-4680-88bb-c69160b37e84 service nova] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:51:01.364932 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1400/2810] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:51:01 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:51:02.595185 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8147102d-f64e-49ef-a08c-95d3282c35ae None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:51:02.595789 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8147102d-f64e-49ef-a08c-95d3282c35ae None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:51:02.595789 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8147102d-f64e-49ef-a08c-95d3282c35ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:51:02.610986 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:51:02.618727 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-8147102d-f64e-49ef-a08c-95d3282c35ae None None] MFA Rules not processed for user `8a3f7b1c624141feab14c67fa39732a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:51:02.620938 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-8147102d-f64e-49ef-a08c-95d3282c35ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:51:02.629644 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1411/2811] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:51:02 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:51:11.360418 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=87,keystone:INSERT=5 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:51:23.219827 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312f429d-3406-4680-88bb-c69160b37e84 service nova] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:51:23.221589 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-312f429d-3406-4680-88bb-c69160b37e84 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:51:23.298713 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312f429d-3406-4680-88bb-c69160b37e84 service nova] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:51:23.303271 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cec175-0ed7-4759-aea0-cbcdd647bbcf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:51:23.304168 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cec175-0ed7-4759-aea0-cbcdd647bbcf service cinder] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:51:23.304490 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cec175-0ed7-4759-aea0-cbcdd647bbcf service cinder] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:51:23.304798 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cec175-0ed7-4759-aea0-cbcdd647bbcf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:51:23.307634 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-15cec175-0ed7-4759-aea0-cbcdd647bbcf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:51:23.347070 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cec175-0ed7-4759-aea0-cbcdd647bbcf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:51:23.349271 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cec175-0ed7-4759-aea0-cbcdd647bbcf service cinder] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:51:23.363357 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1401/2812] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:51:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:51:26.763356 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8147102d-f64e-49ef-a08c-95d3282c35ae None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:51:26.765450 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-8147102d-f64e-49ef-a08c-95d3282c35ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:51:26.834584 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8147102d-f64e-49ef-a08c-95d3282c35ae None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:51:26.839464 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74aeb31-0ca8-4770-a8a8-f633e32c9bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:51:26.840093 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74aeb31-0ca8-4770-a8a8-f633e32c9bf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:51:26.840366 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74aeb31-0ca8-4770-a8a8-f633e32c9bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:51:26.840638 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74aeb31-0ca8-4770-a8a8-f633e32c9bf9 admin admin] PATH_INFO: `/v3/users/26cc26277d9a4d0589c13356cbf158dd` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:51:26.841769 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74aeb31-0ca8-4770-a8a8-f633e32c9bf9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26cc26277d9a4d0589c13356cbf158dd)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:51:26.843694 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74aeb31-0ca8-4770-a8a8-f633e32c9bf9 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:51:26.891940 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-a74aeb31-0ca8-4770-a8a8-f633e32c9bf9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26cc26277d9a4d0589c13356cbf158dd'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:51:26.892731 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-a74aeb31-0ca8-4770-a8a8-f633e32c9bf9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26cc26277d9a4d0589c13356cbf158dd'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:51:26.900331 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-a74aeb31-0ca8-4770-a8a8-f633e32c9bf9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26cc26277d9a4d0589c13356cbf158dd'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:51:26.912100 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-a74aeb31-0ca8-4770-a8a8-f633e32c9bf9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26cc26277d9a4d0589c13356cbf158dd'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:51:26.912800 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-a74aeb31-0ca8-4770-a8a8-f633e32c9bf9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26cc26277d9a4d0589c13356cbf158dd'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:51:26.926347 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1412/2813] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:51:26 2022] DELETE /identity/v3/users/26cc26277d9a4d0589c13356cbf158dd => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:51:26.930996 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6433bf7b-3491-4cf7-9cdb-9eb022756ed6 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:51:26.931537 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6433bf7b-3491-4cf7-9cdb-9eb022756ed6 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:51:26.931651 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6433bf7b-3491-4cf7-9cdb-9eb022756ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:51:26.951371 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-6433bf7b-3491-4cf7-9cdb-9eb022756ed6 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:51:26.952676 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-6433bf7b-3491-4cf7-9cdb-9eb022756ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:51:27.018186 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1402/2814] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:51:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:51:27.029993 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74aeb31-0ca8-4770-a8a8-f633e32c9bf9 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:51:27.031435 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-a74aeb31-0ca8-4770-a8a8-f633e32c9bf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:51:27.092613 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74aeb31-0ca8-4770-a8a8-f633e32c9bf9 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:51:27.096754 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed02b4b6-32e4-4b58-94f8-bf3446c73d4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:51:27.097873 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed02b4b6-32e4-4b58-94f8-bf3446c73d4d service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:51:27.098518 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed02b4b6-32e4-4b58-94f8-bf3446c73d4d service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:51:27.098786 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed02b4b6-32e4-4b58-94f8-bf3446c73d4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:51:27.111074 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed02b4b6-32e4-4b58-94f8-bf3446c73d4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:51:27.112529 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed02b4b6-32e4-4b58-94f8-bf3446c73d4d service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:51:27.122597 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1413/2815] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:51:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:51:27.526130 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6433bf7b-3491-4cf7-9cdb-9eb022756ed6 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:51:27.528518 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-6433bf7b-3491-4cf7-9cdb-9eb022756ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:51:27.541606 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6433bf7b-3491-4cf7-9cdb-9eb022756ed6 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:51:27.546719 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657795df-cce9-440e-a4ab-3c2a0617b3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:51:27.547497 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657795df-cce9-440e-a4ab-3c2a0617b3e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:51:27.547791 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657795df-cce9-440e-a4ab-3c2a0617b3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:51:27.548062 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657795df-cce9-440e-a4ab-3c2a0617b3e3 admin admin] PATH_INFO: `/v3/projects/232013e158a74a14be6868895aa879ef` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:51:27.549276 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657795df-cce9-440e-a4ab-3c2a0617b3e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=232013e158a74a14be6868895aa879ef)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:51:27.551545 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657795df-cce9-440e-a4ab-3c2a0617b3e3 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:51:27.628968 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-657795df-cce9-440e-a4ab-3c2a0617b3e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '232013e158a74a14be6868895aa879ef', 'request_id': 'req-657795df-cce9-440e-a4ab-3c2a0617b3e3'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:51:27.639718 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-657795df-cce9-440e-a4ab-3c2a0617b3e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '232013e158a74a14be6868895aa879ef', 'request_id': 'req-657795df-cce9-440e-a4ab-3c2a0617b3e3'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:51:27.646915 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-657795df-cce9-440e-a4ab-3c2a0617b3e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '232013e158a74a14be6868895aa879ef', 'request_id': 'req-657795df-cce9-440e-a4ab-3c2a0617b3e3'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:51:27.662981 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-657795df-cce9-440e-a4ab-3c2a0617b3e3 admin admin] The token cache is being invalidate because project 232013e158a74a14be6868895aa879ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81618) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:51:27.663295 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-657795df-cce9-440e-a4ab-3c2a0617b3e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:51:27.663970 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-657795df-cce9-440e-a4ab-3c2a0617b3e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:51:27.665694 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1403/2816] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:51:27 2022] DELETE /identity/v3/projects/232013e158a74a14be6868895aa879ef => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:51:37.118038 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:51:37.659197 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=85,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:51:39.677984 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed02b4b6-32e4-4b58-94f8-bf3446c73d4d service neutron] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:51:39.679470 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-ed02b4b6-32e4-4b58-94f8-bf3446c73d4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:51:39.691006 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed02b4b6-32e4-4b58-94f8-bf3446c73d4d service neutron] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:51:39.741885 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d907a98-203b-45bb-bc60-587673ea220e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:51:39.742532 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d907a98-203b-45bb-bc60-587673ea220e service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:51:39.742810 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d907a98-203b-45bb-bc60-587673ea220e service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:51:39.743080 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d907a98-203b-45bb-bc60-587673ea220e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:51:39.744860 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-4d907a98-203b-45bb-bc60-587673ea220e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:51:39.820034 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d907a98-203b-45bb-bc60-587673ea220e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:51:39.821493 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d907a98-203b-45bb-bc60-587673ea220e service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:51:39.832464 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1414/2817] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:51:39 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:51:41.826507 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657795df-cce9-440e-a4ab-3c2a0617b3e3 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:51:41.828311 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-657795df-cce9-440e-a4ab-3c2a0617b3e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:51:41.925517 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657795df-cce9-440e-a4ab-3c2a0617b3e3 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:51:41.934471 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdecba84-258b-4064-85d0-32ce1f211665 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:51:41.935446 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdecba84-258b-4064-85d0-32ce1f211665 service nova] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:51:41.935887 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdecba84-258b-4064-85d0-32ce1f211665 service nova] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:51:41.936319 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdecba84-258b-4064-85d0-32ce1f211665 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:51:41.939209 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-bdecba84-258b-4064-85d0-32ce1f211665 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:51:41.960849 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdecba84-258b-4064-85d0-32ce1f211665 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:51:41.963693 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdecba84-258b-4064-85d0-32ce1f211665 service nova] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:51:41.983663 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1404/2818] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:51:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:51:49.827786 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=50 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:51:51.975134 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=34 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:52:14.511360 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d907a98-203b-45bb-bc60-587673ea220e service neutron] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:52:14.512787 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-4d907a98-203b-45bb-bc60-587673ea220e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:52:14.568936 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d907a98-203b-45bb-bc60-587673ea220e service neutron] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:52:14.572542 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6755ab24-1119-4344-9047-a64d2e6c9a71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:52:14.573138 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6755ab24-1119-4344-9047-a64d2e6c9a71 service cinder] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:52:14.573417 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6755ab24-1119-4344-9047-a64d2e6c9a71 service cinder] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:52:14.573721 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6755ab24-1119-4344-9047-a64d2e6c9a71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:52:14.575548 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-6755ab24-1119-4344-9047-a64d2e6c9a71 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:52:14.634847 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6755ab24-1119-4344-9047-a64d2e6c9a71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:52:14.636807 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6755ab24-1119-4344-9047-a64d2e6c9a71 service cinder] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:52:14.646407 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1415/2819] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:52:14 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:52:24.641787 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=48 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:53:07.667057 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdecba84-258b-4064-85d0-32ce1f211665 service nova] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:53:07.668451 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-bdecba84-258b-4064-85d0-32ce1f211665 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:53:07.794397 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdecba84-258b-4064-85d0-32ce1f211665 service nova] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:53:07.799495 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd8d5bb-4762-4bbd-9d4a-0ac976a5d059 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1a985495ed48a4acc48e2e6fc7f977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:53:07.800102 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd8d5bb-4762-4bbd-9d4a-0ac976a5d059 service placement] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:53:07.800385 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd8d5bb-4762-4bbd-9d4a-0ac976a5d059 service placement] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:53:07.800666 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd8d5bb-4762-4bbd-9d4a-0ac976a5d059 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:53:07.802476 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-bdd8d5bb-4762-4bbd-9d4a-0ac976a5d059 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:53:07.814732 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd8d5bb-4762-4bbd-9d4a-0ac976a5d059 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:53:07.816295 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd8d5bb-4762-4bbd-9d4a-0ac976a5d059 service placement] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:53:07.826699 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1405/2820] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:53:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:53:17.821811 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=54 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:53:38.527046 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6755ab24-1119-4344-9047-a64d2e6c9a71 service cinder] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:53:38.528488 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-6755ab24-1119-4344-9047-a64d2e6c9a71 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:53:38.572228 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6755ab24-1119-4344-9047-a64d2e6c9a71 service cinder] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:53:38.575924 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0f801b-400f-4144-9ee8-d6e3a87175e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:53:38.576460 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f801b-400f-4144-9ee8-d6e3a87175e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:53:38.576739 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f801b-400f-4144-9ee8-d6e3a87175e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:53:38.577012 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f801b-400f-4144-9ee8-d6e3a87175e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:53:38.579223 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-bb0f801b-400f-4144-9ee8-d6e3a87175e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:53:38.592420 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0f801b-400f-4144-9ee8-d6e3a87175e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:53:38.594909 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0f801b-400f-4144-9ee8-d6e3a87175e3 service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:53:38.605824 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1416/2821] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:53:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:53:48.600948 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=20 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:53:50.712424 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd8d5bb-4762-4bbd-9d4a-0ac976a5d059 service placement] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:53:50.735182 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd8d5bb-4762-4bbd-9d4a-0ac976a5d059 service placement] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:53:50.739119 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2578bb-c06a-4057-b8c0-27f84e2e4738 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:53:50.739738 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2578bb-c06a-4057-b8c0-27f84e2e4738 service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:53:50.740020 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2578bb-c06a-4057-b8c0-27f84e2e4738 service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:53:50.740296 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2578bb-c06a-4057-b8c0-27f84e2e4738 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:53:50.742100 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-ff2578bb-c06a-4057-b8c0-27f84e2e4738 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:53:50.752465 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2578bb-c06a-4057-b8c0-27f84e2e4738 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:53:50.753909 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2578bb-c06a-4057-b8c0-27f84e2e4738 service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:53:50.762574 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1406/2822] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:53:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:53:50.866902 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0f801b-400f-4144-9ee8-d6e3a87175e3 service neutron] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:53:50.868399 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-bb0f801b-400f-4144-9ee8-d6e3a87175e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:53:50.879666 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0f801b-400f-4144-9ee8-d6e3a87175e3 service neutron] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:53:50.883528 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401a4515-580f-45ca-bd45-a434f2eebf12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:53:50.884127 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a4515-580f-45ca-bd45-a434f2eebf12 service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:53:50.884408 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a4515-580f-45ca-bd45-a434f2eebf12 service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:53:50.884674 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a4515-580f-45ca-bd45-a434f2eebf12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:53:50.886334 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-401a4515-580f-45ca-bd45-a434f2eebf12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:53:50.900378 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401a4515-580f-45ca-bd45-a434f2eebf12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:53:50.901777 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401a4515-580f-45ca-bd45-a434f2eebf12 service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:53:50.912557 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1417/2823] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:53:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:54:00.759330 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=11 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:54:00.907689 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=6 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:55:13.422358 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2578bb-c06a-4057-b8c0-27f84e2e4738 service neutron] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:55:13.424044 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-ff2578bb-c06a-4057-b8c0-27f84e2e4738 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:55:13.444840 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2578bb-c06a-4057-b8c0-27f84e2e4738 service neutron] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:55:13.450023 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d8ab31-7a48-4ad7-a1d2-4f20081d0259 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:55:13.451224 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d8ab31-7a48-4ad7-a1d2-4f20081d0259 service nova] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:55:13.451512 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d8ab31-7a48-4ad7-a1d2-4f20081d0259 service nova] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:55:13.451791 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d8ab31-7a48-4ad7-a1d2-4f20081d0259 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:55:13.453570 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-d1d8ab31-7a48-4ad7-a1d2-4f20081d0259 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:55:13.466734 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d8ab31-7a48-4ad7-a1d2-4f20081d0259 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:55:13.468383 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d8ab31-7a48-4ad7-a1d2-4f20081d0259 service nova] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:55:13.479943 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1407/2824] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:55:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:55:13.547211 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401a4515-580f-45ca-bd45-a434f2eebf12 service neutron] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:55:13.579066 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401a4515-580f-45ca-bd45-a434f2eebf12 service neutron] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:55:13.585356 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55cbce2-8de8-48dd-a10e-fafa049e4664 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:55:13.586541 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55cbce2-8de8-48dd-a10e-fafa049e4664 service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:55:13.586942 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55cbce2-8de8-48dd-a10e-fafa049e4664 service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:55:13.588602 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55cbce2-8de8-48dd-a10e-fafa049e4664 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:55:13.591066 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-e55cbce2-8de8-48dd-a10e-fafa049e4664 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:55:13.610853 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55cbce2-8de8-48dd-a10e-fafa049e4664 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:55:13.613526 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55cbce2-8de8-48dd-a10e-fafa049e4664 service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:55:13.634214 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1418/2825] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:55:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:55:21.587982 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d8ab31-7a48-4ad7-a1d2-4f20081d0259 service nova] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:55:21.602203 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d8ab31-7a48-4ad7-a1d2-4f20081d0259 service nova] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:55:21.611010 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081d72a2-7e2e-4bd6-837e-472c1b996628 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1a985495ed48a4acc48e2e6fc7f977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:55:21.611726 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d72a2-7e2e-4bd6-837e-472c1b996628 service placement] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:55:21.612151 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d72a2-7e2e-4bd6-837e-472c1b996628 service placement] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:55:21.612630 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d72a2-7e2e-4bd6-837e-472c1b996628 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:55:21.614503 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-081d72a2-7e2e-4bd6-837e-472c1b996628 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:55:21.630778 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081d72a2-7e2e-4bd6-837e-472c1b996628 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:55:21.632554 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081d72a2-7e2e-4bd6-837e-472c1b996628 service placement] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:55:21.644285 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1408/2826] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:55:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:55:23.626858 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=10 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:55:24.735803 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55cbce2-8de8-48dd-a10e-fafa049e4664 service neutron] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:55:24.737350 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-e55cbce2-8de8-48dd-a10e-fafa049e4664 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:55:24.837817 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55cbce2-8de8-48dd-a10e-fafa049e4664 service neutron] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:55:24.843200 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef24bb2-87c3-4a47-80b1-5b14b55526be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:55:24.843827 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef24bb2-87c3-4a47-80b1-5b14b55526be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:55:24.844106 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef24bb2-87c3-4a47-80b1-5b14b55526be admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:55:24.844407 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef24bb2-87c3-4a47-80b1-5b14b55526be admin admin] PATH_INFO: `/v3/users/8a3f7b1c624141feab14c67fa39732a1` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:55:24.845523 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef24bb2-87c3-4a47-80b1-5b14b55526be admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a3f7b1c624141feab14c67fa39732a1)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:55:24.847475 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef24bb2-87c3-4a47-80b1-5b14b55526be admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:55:24.894093 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-cef24bb2-87c3-4a47-80b1-5b14b55526be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a3f7b1c624141feab14c67fa39732a1'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:55:24.894686 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-cef24bb2-87c3-4a47-80b1-5b14b55526be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a3f7b1c624141feab14c67fa39732a1'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:55:24.901596 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-cef24bb2-87c3-4a47-80b1-5b14b55526be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a3f7b1c624141feab14c67fa39732a1'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:55:24.913864 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-cef24bb2-87c3-4a47-80b1-5b14b55526be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a3f7b1c624141feab14c67fa39732a1'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:55:24.914549 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-cef24bb2-87c3-4a47-80b1-5b14b55526be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a3f7b1c624141feab14c67fa39732a1'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:55:24.928007 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1419/2827] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:55:24 2022] DELETE /identity/v3/users/8a3f7b1c624141feab14c67fa39732a1 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:55:25.102750 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081d72a2-7e2e-4bd6-837e-472c1b996628 service placement] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:55:25.119116 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081d72a2-7e2e-4bd6-837e-472c1b996628 service placement] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:55:25.179568 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e7728-7fc1-4da2-bd49-cc381f99edcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:55:25.180111 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e7728-7fc1-4da2-bd49-cc381f99edcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:55:25.180389 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e7728-7fc1-4da2-bd49-cc381f99edcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:55:25.180662 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e7728-7fc1-4da2-bd49-cc381f99edcf admin admin] PATH_INFO: `/v3/projects/b62d50ba5e3d451dab48a4b6cdc50399` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:55:25.182100 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500e7728-7fc1-4da2-bd49-cc381f99edcf admin admin] RBAC: Authorizing `identity:delete_project(project_id=b62d50ba5e3d451dab48a4b6cdc50399)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:55:25.184312 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500e7728-7fc1-4da2-bd49-cc381f99edcf admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:55:25.236486 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-500e7728-7fc1-4da2-bd49-cc381f99edcf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b62d50ba5e3d451dab48a4b6cdc50399', 'request_id': 'req-500e7728-7fc1-4da2-bd49-cc381f99edcf'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:55:25.242833 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-500e7728-7fc1-4da2-bd49-cc381f99edcf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b62d50ba5e3d451dab48a4b6cdc50399', 'request_id': 'req-500e7728-7fc1-4da2-bd49-cc381f99edcf'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:55:25.249311 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-500e7728-7fc1-4da2-bd49-cc381f99edcf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b62d50ba5e3d451dab48a4b6cdc50399', 'request_id': 'req-500e7728-7fc1-4da2-bd49-cc381f99edcf'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:55:25.262422 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-500e7728-7fc1-4da2-bd49-cc381f99edcf admin admin] The token cache is being invalidate because project b62d50ba5e3d451dab48a4b6cdc50399 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81618) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:55:25.262711 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-500e7728-7fc1-4da2-bd49-cc381f99edcf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:55:25.263346 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-500e7728-7fc1-4da2-bd49-cc381f99edcf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:55:25.265344 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1409/2828] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:55:25 2022] DELETE /identity/v3/projects/b62d50ba5e3d451dab48a4b6cdc50399 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:55:25.269494 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef24bb2-87c3-4a47-80b1-5b14b55526be admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:55:25.270963 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-cef24bb2-87c3-4a47-80b1-5b14b55526be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:55:25.284787 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef24bb2-87c3-4a47-80b1-5b14b55526be admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:55:25.339791 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:55:25.354690 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b528c36-97c9-4cb5-b05c-df06c852f680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:55:25.355333 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b528c36-97c9-4cb5-b05c-df06c852f680 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:55:25.355611 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b528c36-97c9-4cb5-b05c-df06c852f680 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:55:25.355908 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b528c36-97c9-4cb5-b05c-df06c852f680 admin admin] PATH_INFO: `/v3/users/9a0f1c6dfd8243b0958a5c7517af5438` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:55:25.357076 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b528c36-97c9-4cb5-b05c-df06c852f680 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a0f1c6dfd8243b0958a5c7517af5438)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:55:25.359079 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b528c36-97c9-4cb5-b05c-df06c852f680 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:55:25.408385 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-4b528c36-97c9-4cb5-b05c-df06c852f680 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a0f1c6dfd8243b0958a5c7517af5438'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:55:25.409028 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-4b528c36-97c9-4cb5-b05c-df06c852f680 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a0f1c6dfd8243b0958a5c7517af5438'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:55:25.416118 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-4b528c36-97c9-4cb5-b05c-df06c852f680 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a0f1c6dfd8243b0958a5c7517af5438'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:55:25.428535 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-4b528c36-97c9-4cb5-b05c-df06c852f680 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a0f1c6dfd8243b0958a5c7517af5438'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:55:25.429259 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-4b528c36-97c9-4cb5-b05c-df06c852f680 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a0f1c6dfd8243b0958a5c7517af5438'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:55:25.443128 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1420/2829] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:55:25 2022] DELETE /identity/v3/users/9a0f1c6dfd8243b0958a5c7517af5438 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:55:25.611460 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e7728-7fc1-4da2-bd49-cc381f99edcf admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:55:25.612878 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-500e7728-7fc1-4da2-bd49-cc381f99edcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:55:25.626590 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e7728-7fc1-4da2-bd49-cc381f99edcf admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:55:25.685389 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb81b88-0791-42ff-a9d6-2deb5345e745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:55:25.686629 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb81b88-0791-42ff-a9d6-2deb5345e745 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:55:25.686903 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb81b88-0791-42ff-a9d6-2deb5345e745 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:55:25.687173 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb81b88-0791-42ff-a9d6-2deb5345e745 admin admin] PATH_INFO: `/v3/projects/27c3d04088f9422492d1549c6663bde4` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:55:25.688273 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb81b88-0791-42ff-a9d6-2deb5345e745 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27c3d04088f9422492d1549c6663bde4)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:55:25.690140 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb81b88-0791-42ff-a9d6-2deb5345e745 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:55:25.696325 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:55:25.738083 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-cbb81b88-0791-42ff-a9d6-2deb5345e745 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27c3d04088f9422492d1549c6663bde4', 'request_id': 'req-cbb81b88-0791-42ff-a9d6-2deb5345e745'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:55:25.745842 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-cbb81b88-0791-42ff-a9d6-2deb5345e745 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27c3d04088f9422492d1549c6663bde4', 'request_id': 'req-cbb81b88-0791-42ff-a9d6-2deb5345e745'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:55:25.752132 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-cbb81b88-0791-42ff-a9d6-2deb5345e745 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27c3d04088f9422492d1549c6663bde4', 'request_id': 'req-cbb81b88-0791-42ff-a9d6-2deb5345e745'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:55:25.765557 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-cbb81b88-0791-42ff-a9d6-2deb5345e745 admin admin] The token cache is being invalidate because project 27c3d04088f9422492d1549c6663bde4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81618) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:55:25.765844 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-cbb81b88-0791-42ff-a9d6-2deb5345e745 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:55:25.766507 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-cbb81b88-0791-42ff-a9d6-2deb5345e745 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:55:25.767977 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1410/2830] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:55:25 2022] DELETE /identity/v3/projects/27c3d04088f9422492d1549c6663bde4 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:55:35.439709 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=30,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:55:35.762208 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=22,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:57:01.241521 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b528c36-97c9-4cb5-b05c-df06c852f680 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:01.243018 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-4b528c36-97c9-4cb5-b05c-df06c852f680 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:01.299364 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b528c36-97c9-4cb5-b05c-df06c852f680 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:01.348077 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969fb0b7-0fbe-4e98-882b-ebcfe9715131 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:01.348591 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969fb0b7-0fbe-4e98-882b-ebcfe9715131 service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:01.348865 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969fb0b7-0fbe-4e98-882b-ebcfe9715131 service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:01.349135 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969fb0b7-0fbe-4e98-882b-ebcfe9715131 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:01.350884 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-969fb0b7-0fbe-4e98-882b-ebcfe9715131 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:01.411442 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969fb0b7-0fbe-4e98-882b-ebcfe9715131 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:01.412852 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969fb0b7-0fbe-4e98-882b-ebcfe9715131 service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:01.422430 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1421/2831] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:57:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:57:04.605497 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb81b88-0791-42ff-a9d6-2deb5345e745 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:04.607859 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-cbb81b88-0791-42ff-a9d6-2deb5345e745 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:04.622871 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb81b88-0791-42ff-a9d6-2deb5345e745 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:04.628186 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b0d645-780d-46f1-ad65-a859f19b8776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:04.628806 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b0d645-780d-46f1-ad65-a859f19b8776 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:04.629080 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b0d645-780d-46f1-ad65-a859f19b8776 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:04.629348 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b0d645-780d-46f1-ad65-a859f19b8776 admin admin] PATH_INFO: `/v3/users/bd190d8e252342ca9232f7a0c966c715` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:04.642889 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b0d645-780d-46f1-ad65-a859f19b8776 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd190d8e252342ca9232f7a0c966c715)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:04.644792 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b0d645-780d-46f1-ad65-a859f19b8776 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:04.692269 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-10b0d645-780d-46f1-ad65-a859f19b8776 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd190d8e252342ca9232f7a0c966c715'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:04.699258 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-10b0d645-780d-46f1-ad65-a859f19b8776 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd190d8e252342ca9232f7a0c966c715'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:04.711431 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-10b0d645-780d-46f1-ad65-a859f19b8776 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd190d8e252342ca9232f7a0c966c715'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:04.719061 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-10b0d645-780d-46f1-ad65-a859f19b8776 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd190d8e252342ca9232f7a0c966c715'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:04.720187 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-10b0d645-780d-46f1-ad65-a859f19b8776 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd190d8e252342ca9232f7a0c966c715'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:04.726698 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1411/2832] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:57:04 2022] DELETE /identity/v3/users/bd190d8e252342ca9232f7a0c966c715 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:57:04.898350 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969fb0b7-0fbe-4e98-882b-ebcfe9715131 service neutron] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:04.910562 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969fb0b7-0fbe-4e98-882b-ebcfe9715131 service neutron] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:04.971473 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f903771-008d-4076-8c3c-9c1e6adff697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:04.972094 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f903771-008d-4076-8c3c-9c1e6adff697 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:04.972369 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f903771-008d-4076-8c3c-9c1e6adff697 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:04.972636 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f903771-008d-4076-8c3c-9c1e6adff697 admin admin] PATH_INFO: `/v3/projects/344bf851ff7d471fba47bde9b6f7e219` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:04.973725 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f903771-008d-4076-8c3c-9c1e6adff697 admin admin] RBAC: Authorizing `identity:delete_project(project_id=344bf851ff7d471fba47bde9b6f7e219)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:04.975636 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f903771-008d-4076-8c3c-9c1e6adff697 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:04.983373 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=100 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:57:05.021910 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-9f903771-008d-4076-8c3c-9c1e6adff697 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '344bf851ff7d471fba47bde9b6f7e219', 'request_id': 'req-9f903771-008d-4076-8c3c-9c1e6adff697'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:05.027970 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-9f903771-008d-4076-8c3c-9c1e6adff697 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '344bf851ff7d471fba47bde9b6f7e219', 'request_id': 'req-9f903771-008d-4076-8c3c-9c1e6adff697'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:05.033972 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-9f903771-008d-4076-8c3c-9c1e6adff697 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '344bf851ff7d471fba47bde9b6f7e219', 'request_id': 'req-9f903771-008d-4076-8c3c-9c1e6adff697'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:05.046949 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-9f903771-008d-4076-8c3c-9c1e6adff697 admin admin] The token cache is being invalidate because project 344bf851ff7d471fba47bde9b6f7e219 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81617) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:57:05.047281 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-9f903771-008d-4076-8c3c-9c1e6adff697 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:05.047910 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-9f903771-008d-4076-8c3c-9c1e6adff697 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:05.049387 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1422/2833] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:57:04 2022] DELETE /identity/v3/projects/344bf851ff7d471fba47bde9b6f7e219 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:57:05.055317 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8361d314-4e12-4534-ba04-90181fbb854e None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:05.055618 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8361d314-4e12-4534-ba04-90181fbb854e None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:05.055875 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8361d314-4e12-4534-ba04-90181fbb854e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:05.114489 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-8361d314-4e12-4534-ba04-90181fbb854e None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:57:05.115721 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-8361d314-4e12-4534-ba04-90181fbb854e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:05.175956 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1412/2834] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:57:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:57:05.184326 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc498ca-3980-4fb6-8dc3-0ca6c6bbd3b2 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:05.184608 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc498ca-3980-4fb6-8dc3-0ca6c6bbd3b2 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:05.184883 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc498ca-3980-4fb6-8dc3-0ca6c6bbd3b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:05.205977 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-7dc498ca-3980-4fb6-8dc3-0ca6c6bbd3b2 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:57:05.207379 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-7dc498ca-3980-4fb6-8dc3-0ca6c6bbd3b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:05.215540 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1423/2835] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:57:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:57:05.220359 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8361d314-4e12-4534-ba04-90181fbb854e None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:05.230226 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8361d314-4e12-4534-ba04-90181fbb854e None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:05.235340 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55f2a7d-061f-4396-8063-c4b52190dfdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:05.235942 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f2a7d-061f-4396-8063-c4b52190dfdb admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:05.236216 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f2a7d-061f-4396-8063-c4b52190dfdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:05.236489 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f2a7d-061f-4396-8063-c4b52190dfdb admin admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:05.237139 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55f2a7d-061f-4396-8063-c4b52190dfdb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:57:05.237437 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55f2a7d-061f-4396-8063-c4b52190dfdb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:05.239057 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55f2a7d-061f-4396-8063-c4b52190dfdb admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:05.251458 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1413/2836] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:57:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:57:05.255968 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc498ca-3980-4fb6-8dc3-0ca6c6bbd3b2 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:05.265272 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc498ca-3980-4fb6-8dc3-0ca6c6bbd3b2 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:05.269958 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7585876d-a8e7-48c9-b97c-e4c3fc376069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:05.270576 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7585876d-a8e7-48c9-b97c-e4c3fc376069 admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:05.270856 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7585876d-a8e7-48c9-b97c-e4c3fc376069 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:05.271124 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7585876d-a8e7-48c9-b97c-e4c3fc376069 admin admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:05.271764 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7585876d-a8e7-48c9-b97c-e4c3fc376069 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:05.273674 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7585876d-a8e7-48c9-b97c-e4c3fc376069 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:05.311413 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1424/2837] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:57:05 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 16:57:05.315809 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55f2a7d-061f-4396-8063-c4b52190dfdb admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:05.326026 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55f2a7d-061f-4396-8063-c4b52190dfdb admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:05.338947 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:57:05.385822 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea22227-dc73-417c-b0a9-c319623e192c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:05.386451 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea22227-dc73-417c-b0a9-c319623e192c admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:05.386736 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea22227-dc73-417c-b0a9-c319623e192c admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:05.387010 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea22227-dc73-417c-b0a9-c319623e192c admin admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:05.387650 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea22227-dc73-417c-b0a9-c319623e192c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:05.389553 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea22227-dc73-417c-b0a9-c319623e192c admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:05.411767 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1414/2838] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:57:05 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:57:05.415868 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7585876d-a8e7-48c9-b97c-e4c3fc376069 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:05.426122 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7585876d-a8e7-48c9-b97c-e4c3fc376069 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:05.430770 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2da7bb2-cac5-4fca-a683-eabe09a16ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:05.431362 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da7bb2-cac5-4fca-a683-eabe09a16ed9 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:05.431641 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da7bb2-cac5-4fca-a683-eabe09a16ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:05.431913 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da7bb2-cac5-4fca-a683-eabe09a16ed9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:05.432551 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2da7bb2-cac5-4fca-a683-eabe09a16ed9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:05.434405 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2da7bb2-cac5-4fca-a683-eabe09a16ed9 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:05.444241 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1425/2839] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:57:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:57:05.448161 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea22227-dc73-417c-b0a9-c319623e192c admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:05.458432 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea22227-dc73-417c-b0a9-c319623e192c admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:05.462964 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26911c40-7558-4aff-87a4-bc0c4ef6ead5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:05.463548 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26911c40-7558-4aff-87a4-bc0c4ef6ead5 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:05.463822 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26911c40-7558-4aff-87a4-bc0c4ef6ead5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:05.464092 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26911c40-7558-4aff-87a4-bc0c4ef6ead5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:05.464718 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26911c40-7558-4aff-87a4-bc0c4ef6ead5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:05.466267 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26911c40-7558-4aff-87a4-bc0c4ef6ead5 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:05.477306 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1415/2840] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:57:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:57:05.481457 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2da7bb2-cac5-4fca-a683-eabe09a16ed9 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:05.491336 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2da7bb2-cac5-4fca-a683-eabe09a16ed9 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:05.495891 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41949454-aaa5-4abb-bbd7-9d2a28aa6a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:05.496538 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41949454-aaa5-4abb-bbd7-9d2a28aa6a0b admin admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:05.496821 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41949454-aaa5-4abb-bbd7-9d2a28aa6a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:05.497100 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41949454-aaa5-4abb-bbd7-9d2a28aa6a0b admin admin] PATH_INFO: `/v3/projects/aefa893439ba4dc48e8a28bda86f9baf/users/d211f57d34fc49c79c89131f1b20c38b/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:05.512649 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41949454-aaa5-4abb-bbd7-9d2a28aa6a0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=aefa893439ba4dc48e8a28bda86f9baf, user_id=d211f57d34fc49c79c89131f1b20c38b, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:05.515899 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41949454-aaa5-4abb-bbd7-9d2a28aa6a0b admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:05.529260 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1426/2841] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:57:05 2022] PUT /identity/v3/projects/aefa893439ba4dc48e8a28bda86f9baf/users/d211f57d34fc49c79c89131f1b20c38b/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:57:05.546423 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d54fb13-5659-4033-827e-3f3ca49a1abf None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:05.546733 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d54fb13-5659-4033-827e-3f3ca49a1abf None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:05.546974 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d54fb13-5659-4033-827e-3f3ca49a1abf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:05.580327 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-2d54fb13-5659-4033-827e-3f3ca49a1abf None None] MFA Rules not processed for user `d211f57d34fc49c79c89131f1b20c38b`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:57:05.581556 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-2d54fb13-5659-4033-827e-3f3ca49a1abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:05.644993 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1416/2842] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:57:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:57:05.657304 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41949454-aaa5-4abb-bbd7-9d2a28aa6a0b admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:05.659015 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-41949454-aaa5-4abb-bbd7-9d2a28aa6a0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:05.748352 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41949454-aaa5-4abb-bbd7-9d2a28aa6a0b admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:05.754190 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abb2f6d-3323-41fd-bee7-d5f3db4166dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:05.754840 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb2f6d-3323-41fd-bee7-d5f3db4166dc service nova] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:05.755115 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb2f6d-3323-41fd-bee7-d5f3db4166dc service nova] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:05.755390 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb2f6d-3323-41fd-bee7-d5f3db4166dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:05.760362 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:57:05.764675 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abb2f6d-3323-41fd-bee7-d5f3db4166dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:05.766765 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abb2f6d-3323-41fd-bee7-d5f3db4166dc service nova] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:05.780220 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1427/2843] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:57:05 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:57:06.105049 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d54fb13-5659-4033-827e-3f3ca49a1abf None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:06.106905 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-2d54fb13-5659-4033-827e-3f3ca49a1abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:06.194596 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d54fb13-5659-4033-827e-3f3ca49a1abf None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:06.198679 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01037acd-3cb2-492d-8dd5-b73f81e0323f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:06.199291 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01037acd-3cb2-492d-8dd5-b73f81e0323f service cinder] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:06.199574 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01037acd-3cb2-492d-8dd5-b73f81e0323f service cinder] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:06.199845 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01037acd-3cb2-492d-8dd5-b73f81e0323f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:06.201613 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-01037acd-3cb2-492d-8dd5-b73f81e0323f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:06.215639 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01037acd-3cb2-492d-8dd5-b73f81e0323f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:06.217042 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01037acd-3cb2-492d-8dd5-b73f81e0323f service cinder] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:06.228244 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1417/2844] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:57:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:57:14.268954 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abb2f6d-3323-41fd-bee7-d5f3db4166dc service nova] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:14.332766 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abb2f6d-3323-41fd-bee7-d5f3db4166dc service nova] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:14.337375 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1582f26-c770-414f-bc3a-f43172d77cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:14.338026 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1582f26-c770-414f-bc3a-f43172d77cf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:14.338293 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1582f26-c770-414f-bc3a-f43172d77cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:14.338625 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1582f26-c770-414f-bc3a-f43172d77cf3 admin admin] PATH_INFO: `/v3/users/d211f57d34fc49c79c89131f1b20c38b` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:14.339766 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1582f26-c770-414f-bc3a-f43172d77cf3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d211f57d34fc49c79c89131f1b20c38b)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:14.341681 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1582f26-c770-414f-bc3a-f43172d77cf3 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:14.389236 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-e1582f26-c770-414f-bc3a-f43172d77cf3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd211f57d34fc49c79c89131f1b20c38b'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:14.390313 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-e1582f26-c770-414f-bc3a-f43172d77cf3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd211f57d34fc49c79c89131f1b20c38b'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:14.397052 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-e1582f26-c770-414f-bc3a-f43172d77cf3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd211f57d34fc49c79c89131f1b20c38b'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:14.409422 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-e1582f26-c770-414f-bc3a-f43172d77cf3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd211f57d34fc49c79c89131f1b20c38b'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:14.410121 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-e1582f26-c770-414f-bc3a-f43172d77cf3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd211f57d34fc49c79c89131f1b20c38b'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:14.424022 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1428/2845] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:57:14 2022] DELETE /identity/v3/users/d211f57d34fc49c79c89131f1b20c38b => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:57:14.429581 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe499ca-3ded-44a3-9a58-4bed49b9aa8c None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:14.430055 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe499ca-3ded-44a3-9a58-4bed49b9aa8c None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:14.430165 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe499ca-3ded-44a3-9a58-4bed49b9aa8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:14.450985 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-ffe499ca-3ded-44a3-9a58-4bed49b9aa8c None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:57:14.452211 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-ffe499ca-3ded-44a3-9a58-4bed49b9aa8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:14.456934 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:57:14.514574 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1418/2846] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:57:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:57:14.525022 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1582f26-c770-414f-bc3a-f43172d77cf3 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:14.526574 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-e1582f26-c770-414f-bc3a-f43172d77cf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:14.586170 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1582f26-c770-414f-bc3a-f43172d77cf3 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:14.589955 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9cc9b9-f139-49e9-bdba-63a4bc0d7936 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:14.591651 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9cc9b9-f139-49e9-bdba-63a4bc0d7936 service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:14.591651 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9cc9b9-f139-49e9-bdba-63a4bc0d7936 service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:14.591651 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9cc9b9-f139-49e9-bdba-63a4bc0d7936 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:14.601282 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9cc9b9-f139-49e9-bdba-63a4bc0d7936 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:14.602744 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9cc9b9-f139-49e9-bdba-63a4bc0d7936 service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:14.612526 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1429/2847] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:57:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:57:14.910149 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe499ca-3ded-44a3-9a58-4bed49b9aa8c None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:14.911791 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-ffe499ca-3ded-44a3-9a58-4bed49b9aa8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:14.928322 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe499ca-3ded-44a3-9a58-4bed49b9aa8c None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:14.936205 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fc04e6-058f-46e8-ab3e-1882ef8bc549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:14.937521 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fc04e6-058f-46e8-ab3e-1882ef8bc549 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:14.937990 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fc04e6-058f-46e8-ab3e-1882ef8bc549 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:14.938249 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fc04e6-058f-46e8-ab3e-1882ef8bc549 admin admin] PATH_INFO: `/v3/projects/aefa893439ba4dc48e8a28bda86f9baf` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:14.940670 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fc04e6-058f-46e8-ab3e-1882ef8bc549 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aefa893439ba4dc48e8a28bda86f9baf)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:14.943730 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fc04e6-058f-46e8-ab3e-1882ef8bc549 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:15.011067 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-98fc04e6-058f-46e8-ab3e-1882ef8bc549 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aefa893439ba4dc48e8a28bda86f9baf', 'request_id': 'req-98fc04e6-058f-46e8-ab3e-1882ef8bc549'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:15.018809 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-98fc04e6-058f-46e8-ab3e-1882ef8bc549 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aefa893439ba4dc48e8a28bda86f9baf', 'request_id': 'req-98fc04e6-058f-46e8-ab3e-1882ef8bc549'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:15.025075 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-98fc04e6-058f-46e8-ab3e-1882ef8bc549 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aefa893439ba4dc48e8a28bda86f9baf', 'request_id': 'req-98fc04e6-058f-46e8-ab3e-1882ef8bc549'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:15.039557 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-98fc04e6-058f-46e8-ab3e-1882ef8bc549 admin admin] The token cache is being invalidate because project aefa893439ba4dc48e8a28bda86f9baf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81618) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:57:15.039850 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-98fc04e6-058f-46e8-ab3e-1882ef8bc549 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:15.040508 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-98fc04e6-058f-46e8-ab3e-1882ef8bc549 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:15.042723 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1419/2848] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:57:14 2022] DELETE /identity/v3/projects/aefa893439ba4dc48e8a28bda86f9baf => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 16:57:15.048165 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f10297c-a046-4655-9811-fdc136fe8118 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:15.048468 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f10297c-a046-4655-9811-fdc136fe8118 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:15.048747 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f10297c-a046-4655-9811-fdc136fe8118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:15.069708 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-3f10297c-a046-4655-9811-fdc136fe8118 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:57:15.070895 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-3f10297c-a046-4655-9811-fdc136fe8118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:15.082655 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:57:15.147529 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1430/2849] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:57:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:57:15.155649 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa45fd9-d30b-4d5d-92dc-135bc14cf95e None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:15.155930 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa45fd9-d30b-4d5d-92dc-135bc14cf95e None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:15.156203 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa45fd9-d30b-4d5d-92dc-135bc14cf95e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:15.176192 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-0fa45fd9-d30b-4d5d-92dc-135bc14cf95e None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:57:15.177413 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-0fa45fd9-d30b-4d5d-92dc-135bc14cf95e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:15.185587 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1420/2850] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:57:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:57:15.190605 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f10297c-a046-4655-9811-fdc136fe8118 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:15.200700 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f10297c-a046-4655-9811-fdc136fe8118 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:15.205283 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09634bc4-6dc2-42d3-a22c-681b66f4fdf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:15.205945 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09634bc4-6dc2-42d3-a22c-681b66f4fdf0 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:15.206212 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09634bc4-6dc2-42d3-a22c-681b66f4fdf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:15.206521 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09634bc4-6dc2-42d3-a22c-681b66f4fdf0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:15.207167 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09634bc4-6dc2-42d3-a22c-681b66f4fdf0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:57:15.207470 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09634bc4-6dc2-42d3-a22c-681b66f4fdf0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:15.209054 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09634bc4-6dc2-42d3-a22c-681b66f4fdf0 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:15.220714 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1431/2851] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:57:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:57:15.225272 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa45fd9-d30b-4d5d-92dc-135bc14cf95e None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:15.235334 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa45fd9-d30b-4d5d-92dc-135bc14cf95e None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:15.240184 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c0a3a5-4456-405c-82a7-2387b8e9d638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:15.240836 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0a3a5-4456-405c-82a7-2387b8e9d638 admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:15.241305 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0a3a5-4456-405c-82a7-2387b8e9d638 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:15.241781 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0a3a5-4456-405c-82a7-2387b8e9d638 admin admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:15.242712 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c0a3a5-4456-405c-82a7-2387b8e9d638 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:15.244829 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c0a3a5-4456-405c-82a7-2387b8e9d638 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:15.279383 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1421/2852] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:57:15 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:57:15.283979 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09634bc4-6dc2-42d3-a22c-681b66f4fdf0 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:15.296586 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09634bc4-6dc2-42d3-a22c-681b66f4fdf0 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:15.360697 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58d0646-e085-4139-a241-2e5dc6309a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:15.361521 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d0646-e085-4139-a241-2e5dc6309a66 admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:15.361634 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d0646-e085-4139-a241-2e5dc6309a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:15.361846 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d0646-e085-4139-a241-2e5dc6309a66 admin admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:15.362527 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58d0646-e085-4139-a241-2e5dc6309a66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:15.364506 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58d0646-e085-4139-a241-2e5dc6309a66 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:15.389880 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1432/2853] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:57:15 2022] POST /identity/v3/users => generated 286 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:57:15.394557 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c0a3a5-4456-405c-82a7-2387b8e9d638 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:15.405037 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c0a3a5-4456-405c-82a7-2387b8e9d638 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:15.409829 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d88ad5-9220-4613-99f8-46146a059bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:15.410505 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d88ad5-9220-4613-99f8-46146a059bd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:15.410791 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d88ad5-9220-4613-99f8-46146a059bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:15.411074 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d88ad5-9220-4613-99f8-46146a059bd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:15.411719 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d88ad5-9220-4613-99f8-46146a059bd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:15.413393 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d88ad5-9220-4613-99f8-46146a059bd9 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:15.425475 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1422/2854] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:57:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:57:15.429795 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58d0646-e085-4139-a241-2e5dc6309a66 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:15.440709 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58d0646-e085-4139-a241-2e5dc6309a66 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:15.445363 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b897ced1-d7e8-4726-83a3-6c677db12176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:15.445952 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b897ced1-d7e8-4726-83a3-6c677db12176 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:15.446227 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b897ced1-d7e8-4726-83a3-6c677db12176 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:15.446559 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b897ced1-d7e8-4726-83a3-6c677db12176 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:15.447282 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b897ced1-d7e8-4726-83a3-6c677db12176 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:15.448899 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b897ced1-d7e8-4726-83a3-6c677db12176 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:15.459074 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1433/2855] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:57:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:57:15.463365 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d88ad5-9220-4613-99f8-46146a059bd9 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:15.475134 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d88ad5-9220-4613-99f8-46146a059bd9 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:15.480275 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25add9c-6417-455a-a91b-50859715affa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:15.480975 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25add9c-6417-455a-a91b-50859715affa admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:15.481295 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25add9c-6417-455a-a91b-50859715affa admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:15.481631 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25add9c-6417-455a-a91b-50859715affa admin admin] PATH_INFO: `/v3/projects/f1a09fac0d9344fc84e0d7d23a252c82/users/06217a545f714b509145f0ce4dd62ad9/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:15.498432 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25add9c-6417-455a-a91b-50859715affa admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1a09fac0d9344fc84e0d7d23a252c82, user_id=06217a545f714b509145f0ce4dd62ad9, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:15.501087 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25add9c-6417-455a-a91b-50859715affa admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:15.515334 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1423/2856] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:57:15 2022] PUT /identity/v3/projects/f1a09fac0d9344fc84e0d7d23a252c82/users/06217a545f714b509145f0ce4dd62ad9/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:57:15.526262 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff30aff2-7f57-4732-9308-7a498f9a50d1 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:15.526573 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff30aff2-7f57-4732-9308-7a498f9a50d1 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:15.526849 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff30aff2-7f57-4732-9308-7a498f9a50d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:15.556093 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-ff30aff2-7f57-4732-9308-7a498f9a50d1 None None] MFA Rules not processed for user `06217a545f714b509145f0ce4dd62ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:57:15.557224 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-ff30aff2-7f57-4732-9308-7a498f9a50d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:15.624525 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1434/2857] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:57:15 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:57:15.635487 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25add9c-6417-455a-a91b-50859715affa admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:15.637228 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-c25add9c-6417-455a-a91b-50859715affa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:15.669109 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:57:15.713063 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25add9c-6417-455a-a91b-50859715affa admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:15.719441 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081411d8-a3b3-47af-8607-56379f3f3df2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecd01cc72834b72b5ff2c4b47249737', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:15.720046 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081411d8-a3b3-47af-8607-56379f3f3df2 service nova] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:15.720325 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081411d8-a3b3-47af-8607-56379f3f3df2 service nova] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:15.720597 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081411d8-a3b3-47af-8607-56379f3f3df2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:15.728381 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081411d8-a3b3-47af-8607-56379f3f3df2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:15.729943 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081411d8-a3b3-47af-8607-56379f3f3df2 service nova] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:15.739547 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1424/2858] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:57:15 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:57:18.516216 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff30aff2-7f57-4732-9308-7a498f9a50d1 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:18.528818 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:57:18.587338 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff30aff2-7f57-4732-9308-7a498f9a50d1 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:18.591881 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a25cd91-7c39-42a9-9fdb-52acf2fb6b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:18.592509 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a25cd91-7c39-42a9-9fdb-52acf2fb6b5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:18.592787 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a25cd91-7c39-42a9-9fdb-52acf2fb6b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:18.593563 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a25cd91-7c39-42a9-9fdb-52acf2fb6b5d admin admin] PATH_INFO: `/v3/users/06217a545f714b509145f0ce4dd62ad9` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:18.594751 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a25cd91-7c39-42a9-9fdb-52acf2fb6b5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=06217a545f714b509145f0ce4dd62ad9)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:18.596669 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a25cd91-7c39-42a9-9fdb-52acf2fb6b5d admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:18.645363 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-6a25cd91-7c39-42a9-9fdb-52acf2fb6b5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06217a545f714b509145f0ce4dd62ad9'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:18.646025 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-6a25cd91-7c39-42a9-9fdb-52acf2fb6b5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06217a545f714b509145f0ce4dd62ad9'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:18.653011 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-6a25cd91-7c39-42a9-9fdb-52acf2fb6b5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06217a545f714b509145f0ce4dd62ad9'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:18.666101 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-6a25cd91-7c39-42a9-9fdb-52acf2fb6b5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06217a545f714b509145f0ce4dd62ad9'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:18.666824 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-6a25cd91-7c39-42a9-9fdb-52acf2fb6b5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06217a545f714b509145f0ce4dd62ad9'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:18.680551 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1435/2859] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:57:18 2022] DELETE /identity/v3/users/06217a545f714b509145f0ce4dd62ad9 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:57:18.687758 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdb2fae-af77-4df3-8495-edf532669291 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:18.688297 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdb2fae-af77-4df3-8495-edf532669291 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:18.688297 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdb2fae-af77-4df3-8495-edf532669291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:18.708595 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-2bdb2fae-af77-4df3-8495-edf532669291 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:57:18.709806 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-2bdb2fae-af77-4df3-8495-edf532669291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:18.774476 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1425/2860] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:57:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:57:18.785703 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a25cd91-7c39-42a9-9fdb-52acf2fb6b5d admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:18.787130 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-6a25cd91-7c39-42a9-9fdb-52acf2fb6b5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:18.846582 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a25cd91-7c39-42a9-9fdb-52acf2fb6b5d admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:18.850243 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deb1bb6-c511-4aba-881c-8eda5d10c156 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:18.850875 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deb1bb6-c511-4aba-881c-8eda5d10c156 service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:18.851184 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deb1bb6-c511-4aba-881c-8eda5d10c156 service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:18.851456 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deb1bb6-c511-4aba-881c-8eda5d10c156 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:18.860722 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deb1bb6-c511-4aba-881c-8eda5d10c156 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:18.862155 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deb1bb6-c511-4aba-881c-8eda5d10c156 service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:18.871896 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1436/2861] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:57:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:57:19.177864 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdb2fae-af77-4df3-8495-edf532669291 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:19.179604 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-2bdb2fae-af77-4df3-8495-edf532669291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:19.200651 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdb2fae-af77-4df3-8495-edf532669291 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:19.218158 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ac035d-ae39-411c-ac6b-e06f12d5296f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:19.218857 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ac035d-ae39-411c-ac6b-e06f12d5296f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:19.219145 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ac035d-ae39-411c-ac6b-e06f12d5296f admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:19.219448 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ac035d-ae39-411c-ac6b-e06f12d5296f admin admin] PATH_INFO: `/v3/projects/f1a09fac0d9344fc84e0d7d23a252c82` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:19.220720 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ac035d-ae39-411c-ac6b-e06f12d5296f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1a09fac0d9344fc84e0d7d23a252c82)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:19.223242 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ac035d-ae39-411c-ac6b-e06f12d5296f admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:19.279478 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-57ac035d-ae39-411c-ac6b-e06f12d5296f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1a09fac0d9344fc84e0d7d23a252c82', 'request_id': 'req-57ac035d-ae39-411c-ac6b-e06f12d5296f'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:19.285899 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-57ac035d-ae39-411c-ac6b-e06f12d5296f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1a09fac0d9344fc84e0d7d23a252c82', 'request_id': 'req-57ac035d-ae39-411c-ac6b-e06f12d5296f'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:19.291905 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-57ac035d-ae39-411c-ac6b-e06f12d5296f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1a09fac0d9344fc84e0d7d23a252c82', 'request_id': 'req-57ac035d-ae39-411c-ac6b-e06f12d5296f'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:19.305749 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-57ac035d-ae39-411c-ac6b-e06f12d5296f admin admin] The token cache is being invalidate because project f1a09fac0d9344fc84e0d7d23a252c82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81618) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:57:19.306025 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-57ac035d-ae39-411c-ac6b-e06f12d5296f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:19.306691 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-57ac035d-ae39-411c-ac6b-e06f12d5296f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:19.308392 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1426/2862] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:57:19 2022] DELETE /identity/v3/projects/f1a09fac0d9344fc84e0d7d23a252c82 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:57:19.314088 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e13501-5907-4201-a89d-fb294c27d1f9 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:19.314393 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e13501-5907-4201-a89d-fb294c27d1f9 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:19.314672 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e13501-5907-4201-a89d-fb294c27d1f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:19.340469 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-e3e13501-5907-4201-a89d-fb294c27d1f9 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:57:19.341881 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-e3e13501-5907-4201-a89d-fb294c27d1f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:19.376980 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:57:19.417148 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1437/2863] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:57:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:57:19.426670 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff13018-e01b-4425-9bca-dec8e8884b62 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:19.426959 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff13018-e01b-4425-9bca-dec8e8884b62 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:19.427237 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff13018-e01b-4425-9bca-dec8e8884b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:19.448192 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-8ff13018-e01b-4425-9bca-dec8e8884b62 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:57:19.449424 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-8ff13018-e01b-4425-9bca-dec8e8884b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:19.458079 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1427/2864] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:57:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:57:19.462547 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e13501-5907-4201-a89d-fb294c27d1f9 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:19.472807 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e13501-5907-4201-a89d-fb294c27d1f9 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:19.477747 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b93c36b-e1fe-4fa2-8c02-48fb6d1b50c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:19.478395 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b93c36b-e1fe-4fa2-8c02-48fb6d1b50c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:19.478682 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b93c36b-e1fe-4fa2-8c02-48fb6d1b50c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:19.478956 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b93c36b-e1fe-4fa2-8c02-48fb6d1b50c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:19.479615 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b93c36b-e1fe-4fa2-8c02-48fb6d1b50c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:57:19.479922 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b93c36b-e1fe-4fa2-8c02-48fb6d1b50c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:19.481514 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b93c36b-e1fe-4fa2-8c02-48fb6d1b50c1 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:19.494392 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1438/2865] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:57:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:57:19.499628 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff13018-e01b-4425-9bca-dec8e8884b62 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:19.509218 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff13018-e01b-4425-9bca-dec8e8884b62 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:19.513960 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c823d13-d527-4d4f-98fa-b0b74a350cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:19.514720 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c823d13-d527-4d4f-98fa-b0b74a350cf7 admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:19.515549 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c823d13-d527-4d4f-98fa-b0b74a350cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:19.515779 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c823d13-d527-4d4f-98fa-b0b74a350cf7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:19.516440 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c823d13-d527-4d4f-98fa-b0b74a350cf7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:19.518333 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c823d13-d527-4d4f-98fa-b0b74a350cf7 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:19.552913 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1428/2866] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:57:19 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:57:19.556643 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b93c36b-e1fe-4fa2-8c02-48fb6d1b50c1 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:19.568374 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b93c36b-e1fe-4fa2-8c02-48fb6d1b50c1 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:19.628435 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5e8c74-ff70-4dcc-a9fd-1c3a5541e31b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:19.629040 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5e8c74-ff70-4dcc-a9fd-1c3a5541e31b admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:19.629323 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5e8c74-ff70-4dcc-a9fd-1c3a5541e31b admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:19.629600 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5e8c74-ff70-4dcc-a9fd-1c3a5541e31b admin admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:19.630248 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5e8c74-ff70-4dcc-a9fd-1c3a5541e31b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:19.632214 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5e8c74-ff70-4dcc-a9fd-1c3a5541e31b admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:19.653373 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1439/2867] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:57:19 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:57:19.657853 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c823d13-d527-4d4f-98fa-b0b74a350cf7 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:19.668153 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c823d13-d527-4d4f-98fa-b0b74a350cf7 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:19.673338 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39286f1-8173-494c-91f2-96a304316de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:19.673939 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39286f1-8173-494c-91f2-96a304316de0 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:19.674222 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39286f1-8173-494c-91f2-96a304316de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:19.674534 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39286f1-8173-494c-91f2-96a304316de0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:19.675177 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39286f1-8173-494c-91f2-96a304316de0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:19.676766 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39286f1-8173-494c-91f2-96a304316de0 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:19.679257 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:57:19.689741 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1429/2868] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:57:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:57:19.693836 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5e8c74-ff70-4dcc-a9fd-1c3a5541e31b admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:19.704128 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5e8c74-ff70-4dcc-a9fd-1c3a5541e31b admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:19.708868 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d42b99-ec3d-4e9e-8a36-5a70ecc4877a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:19.709479 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d42b99-ec3d-4e9e-8a36-5a70ecc4877a admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:19.709760 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d42b99-ec3d-4e9e-8a36-5a70ecc4877a admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:19.710035 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d42b99-ec3d-4e9e-8a36-5a70ecc4877a admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:19.710727 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d42b99-ec3d-4e9e-8a36-5a70ecc4877a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:19.712326 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d42b99-ec3d-4e9e-8a36-5a70ecc4877a admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:19.724462 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1440/2869] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:57:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:57:19.728850 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39286f1-8173-494c-91f2-96a304316de0 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:19.739608 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39286f1-8173-494c-91f2-96a304316de0 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:19.744310 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fce3a61-be5b-443c-8ec6-96a5fc92f13c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:19.744960 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce3a61-be5b-443c-8ec6-96a5fc92f13c admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:19.745241 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce3a61-be5b-443c-8ec6-96a5fc92f13c admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:19.745519 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce3a61-be5b-443c-8ec6-96a5fc92f13c admin admin] PATH_INFO: `/v3/projects/ea4da625d0f34b628068fd1300d971ab/users/d91c9d6ff1b146b6b5c3b7242260038e/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:19.762333 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fce3a61-be5b-443c-8ec6-96a5fc92f13c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea4da625d0f34b628068fd1300d971ab, user_id=d91c9d6ff1b146b6b5c3b7242260038e, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:19.765514 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fce3a61-be5b-443c-8ec6-96a5fc92f13c admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:19.780400 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1430/2870] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:57:19 2022] PUT /identity/v3/projects/ea4da625d0f34b628068fd1300d971ab/users/d91c9d6ff1b146b6b5c3b7242260038e/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:57:19.848551 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36a0bb9-d2a3-498d-8d87-52214c7ba2d5 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:19.848747 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36a0bb9-d2a3-498d-8d87-52214c7ba2d5 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:19.849025 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36a0bb9-d2a3-498d-8d87-52214c7ba2d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:19.878498 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-e36a0bb9-d2a3-498d-8d87-52214c7ba2d5 None None] MFA Rules not processed for user `d91c9d6ff1b146b6b5c3b7242260038e`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:57:19.879745 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-e36a0bb9-d2a3-498d-8d87-52214c7ba2d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:19.946945 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1441/2871] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:57:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:57:19.951452 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fce3a61-be5b-443c-8ec6-96a5fc92f13c admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:20.016146 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fce3a61-be5b-443c-8ec6-96a5fc92f13c admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:20.021133 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bea9396-614c-495d-b1cb-06876362c6a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:20.021744 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bea9396-614c-495d-b1cb-06876362c6a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:20.022024 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bea9396-614c-495d-b1cb-06876362c6a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:20.022296 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bea9396-614c-495d-b1cb-06876362c6a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:20.023000 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bea9396-614c-495d-b1cb-06876362c6a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:20.024896 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bea9396-614c-495d-b1cb-06876362c6a5 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:20.054439 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1431/2872] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:57:19 2022] POST /identity/v3/projects => generated 373 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 16:57:20.058643 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36a0bb9-d2a3-498d-8d87-52214c7ba2d5 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:20.069974 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36a0bb9-d2a3-498d-8d87-52214c7ba2d5 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:20.103947 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:57:20.146496 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df43a3c-5e6d-4e24-a2d1-a45e8840f15a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:20.147408 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df43a3c-5e6d-4e24-a2d1-a45e8840f15a admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:20.147889 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df43a3c-5e6d-4e24-a2d1-a45e8840f15a admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:20.148216 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df43a3c-5e6d-4e24-a2d1-a45e8840f15a admin admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:20.148939 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df43a3c-5e6d-4e24-a2d1-a45e8840f15a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:20.150913 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df43a3c-5e6d-4e24-a2d1-a45e8840f15a admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:20.174294 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1442/2873] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:57:20 2022] POST /identity/v3/users => generated 293 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:57:20.178320 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bea9396-614c-495d-b1cb-06876362c6a5 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:20.189103 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bea9396-614c-495d-b1cb-06876362c6a5 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:20.193872 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3245a1a5-f2f3-4bc4-9658-c0403a639f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:20.194509 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3245a1a5-f2f3-4bc4-9658-c0403a639f4a admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:20.194796 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3245a1a5-f2f3-4bc4-9658-c0403a639f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:20.195074 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3245a1a5-f2f3-4bc4-9658-c0403a639f4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:20.195726 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3245a1a5-f2f3-4bc4-9658-c0403a639f4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:20.197313 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3245a1a5-f2f3-4bc4-9658-c0403a639f4a admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:20.208416 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1432/2874] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:57:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:57:20.212875 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df43a3c-5e6d-4e24-a2d1-a45e8840f15a admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:20.224985 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df43a3c-5e6d-4e24-a2d1-a45e8840f15a admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:20.230190 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b28b789-28ef-4dff-8a48-dabd86e609fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:20.230901 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b28b789-28ef-4dff-8a48-dabd86e609fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:20.231184 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b28b789-28ef-4dff-8a48-dabd86e609fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:20.231462 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b28b789-28ef-4dff-8a48-dabd86e609fb admin admin] PATH_INFO: `/v3/projects/b0cc985c9eed4295975b16bef69fd914/users/cd946315d76b473e8e098530b8fe22d8/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:20.247778 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b28b789-28ef-4dff-8a48-dabd86e609fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0cc985c9eed4295975b16bef69fd914, user_id=cd946315d76b473e8e098530b8fe22d8, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:20.250484 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b28b789-28ef-4dff-8a48-dabd86e609fb admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:20.263884 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1443/2875] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:57:20 2022] PUT /identity/v3/projects/b0cc985c9eed4295975b16bef69fd914/users/cd946315d76b473e8e098530b8fe22d8/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:57:20.267894 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3245a1a5-f2f3-4bc4-9658-c0403a639f4a admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:20.278180 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3245a1a5-f2f3-4bc4-9658-c0403a639f4a admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:20.341607 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547ae3c1-fe4f-4966-aa20-66c13abb15f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:20.342222 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ae3c1-fe4f-4966-aa20-66c13abb15f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:20.342536 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ae3c1-fe4f-4966-aa20-66c13abb15f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:20.342850 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ae3c1-fe4f-4966-aa20-66c13abb15f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:20.344014 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547ae3c1-fe4f-4966-aa20-66c13abb15f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:20.345683 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547ae3c1-fe4f-4966-aa20-66c13abb15f6 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:20.356794 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1433/2876] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:57:20 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 16:57:20.360919 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b28b789-28ef-4dff-8a48-dabd86e609fb admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:20.371446 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b28b789-28ef-4dff-8a48-dabd86e609fb admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:20.376474 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdc206d-b6aa-4a1a-9307-42d4cc8eb3f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:20.377141 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc206d-b6aa-4a1a-9307-42d4cc8eb3f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:20.377416 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc206d-b6aa-4a1a-9307-42d4cc8eb3f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:20.377685 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc206d-b6aa-4a1a-9307-42d4cc8eb3f4 admin admin] PATH_INFO: `/v3/projects/b0cc985c9eed4295975b16bef69fd914/users/cd946315d76b473e8e098530b8fe22d8/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:20.379632 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdc206d-b6aa-4a1a-9307-42d4cc8eb3f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0cc985c9eed4295975b16bef69fd914, user_id=cd946315d76b473e8e098530b8fe22d8, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:20.382229 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdc206d-b6aa-4a1a-9307-42d4cc8eb3f4 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:20.394673 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1444/2877] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:57:20 2022] PUT /identity/v3/projects/b0cc985c9eed4295975b16bef69fd914/users/cd946315d76b473e8e098530b8fe22d8/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:57:20.408369 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80375a5-7da0-4e80-845b-0fae01fabf5c None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:20.408656 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80375a5-7da0-4e80-845b-0fae01fabf5c None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:20.408956 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80375a5-7da0-4e80-845b-0fae01fabf5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:20.438153 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-c80375a5-7da0-4e80-845b-0fae01fabf5c None None] MFA Rules not processed for user `cd946315d76b473e8e098530b8fe22d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:57:20.439471 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-c80375a5-7da0-4e80-845b-0fae01fabf5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:20.473336 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:57:20.509638 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1434/2878] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:57:20 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:57:20.521657 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdc206d-b6aa-4a1a-9307-42d4cc8eb3f4 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:20.523428 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-8cdc206d-b6aa-4a1a-9307-42d4cc8eb3f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:20.585478 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdc206d-b6aa-4a1a-9307-42d4cc8eb3f4 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:20.589329 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8df5a7-ac02-41c7-b76b-5af3f835e3ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:20.589952 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8df5a7-ac02-41c7-b76b-5af3f835e3ea service cinder] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:20.590224 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8df5a7-ac02-41c7-b76b-5af3f835e3ea service cinder] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:20.590539 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8df5a7-ac02-41c7-b76b-5af3f835e3ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:20.600562 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8df5a7-ac02-41c7-b76b-5af3f835e3ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:20.602002 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8df5a7-ac02-41c7-b76b-5af3f835e3ea service cinder] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:20.612255 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1445/2879] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:57:20 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:57:21.021621 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80375a5-7da0-4e80-845b-0fae01fabf5c None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:21.084771 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80375a5-7da0-4e80-845b-0fae01fabf5c None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:21.089313 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e159e0-1eb8-49da-9ec0-f5462934e07e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:21.089945 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e159e0-1eb8-49da-9ec0-f5462934e07e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:21.090220 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e159e0-1eb8-49da-9ec0-f5462934e07e admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:21.090526 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e159e0-1eb8-49da-9ec0-f5462934e07e admin admin] PATH_INFO: `/v3/users/d91c9d6ff1b146b6b5c3b7242260038e` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:21.091661 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e159e0-1eb8-49da-9ec0-f5462934e07e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d91c9d6ff1b146b6b5c3b7242260038e)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:21.093614 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e159e0-1eb8-49da-9ec0-f5462934e07e admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:21.143649 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-f7e159e0-1eb8-49da-9ec0-f5462934e07e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd91c9d6ff1b146b6b5c3b7242260038e'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:21.150526 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-f7e159e0-1eb8-49da-9ec0-f5462934e07e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd91c9d6ff1b146b6b5c3b7242260038e'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:21.162242 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-f7e159e0-1eb8-49da-9ec0-f5462934e07e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd91c9d6ff1b146b6b5c3b7242260038e'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:21.169932 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-f7e159e0-1eb8-49da-9ec0-f5462934e07e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd91c9d6ff1b146b6b5c3b7242260038e'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:21.170660 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-f7e159e0-1eb8-49da-9ec0-f5462934e07e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd91c9d6ff1b146b6b5c3b7242260038e'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:21.177163 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1435/2880] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:57:21 2022] DELETE /identity/v3/users/d91c9d6ff1b146b6b5c3b7242260038e => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:57:21.181792 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce7eecd-d514-43b4-87db-bf203e199f5c None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:21.182076 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce7eecd-d514-43b4-87db-bf203e199f5c None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:21.182397 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce7eecd-d514-43b4-87db-bf203e199f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:21.202504 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-3ce7eecd-d514-43b4-87db-bf203e199f5c None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:57:21.203730 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-3ce7eecd-d514-43b4-87db-bf203e199f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:21.264365 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1446/2881] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:57:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:57:21.274137 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e159e0-1eb8-49da-9ec0-f5462934e07e admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:21.275587 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-f7e159e0-1eb8-49da-9ec0-f5462934e07e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:21.331158 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:57:21.335417 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e159e0-1eb8-49da-9ec0-f5462934e07e admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:21.343086 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c65ae2-9699-4b91-ae15-28beea12140c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:21.343865 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c65ae2-9699-4b91-ae15-28beea12140c service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:21.344149 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c65ae2-9699-4b91-ae15-28beea12140c service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:21.344448 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c65ae2-9699-4b91-ae15-28beea12140c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:21.354179 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c65ae2-9699-4b91-ae15-28beea12140c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:21.355652 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c65ae2-9699-4b91-ae15-28beea12140c service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:21.365541 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1436/2882] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:57:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:57:21.631712 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce7eecd-d514-43b4-87db-bf203e199f5c None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:21.633156 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-3ce7eecd-d514-43b4-87db-bf203e199f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:21.645512 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce7eecd-d514-43b4-87db-bf203e199f5c None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:21.651141 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b0efda-e3e1-4984-85f5-d489984be518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:21.652017 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b0efda-e3e1-4984-85f5-d489984be518 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:21.652427 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b0efda-e3e1-4984-85f5-d489984be518 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:21.652826 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b0efda-e3e1-4984-85f5-d489984be518 admin admin] PATH_INFO: `/v3/projects/ea4da625d0f34b628068fd1300d971ab` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:21.654208 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b0efda-e3e1-4984-85f5-d489984be518 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea4da625d0f34b628068fd1300d971ab)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:21.658237 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b0efda-e3e1-4984-85f5-d489984be518 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:21.685416 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:57:21.712959 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-e2b0efda-e3e1-4984-85f5-d489984be518 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea4da625d0f34b628068fd1300d971ab', 'request_id': 'req-e2b0efda-e3e1-4984-85f5-d489984be518'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:21.719961 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-e2b0efda-e3e1-4984-85f5-d489984be518 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea4da625d0f34b628068fd1300d971ab', 'request_id': 'req-e2b0efda-e3e1-4984-85f5-d489984be518'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:21.725975 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-e2b0efda-e3e1-4984-85f5-d489984be518 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea4da625d0f34b628068fd1300d971ab', 'request_id': 'req-e2b0efda-e3e1-4984-85f5-d489984be518'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:21.740076 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-e2b0efda-e3e1-4984-85f5-d489984be518 admin admin] The token cache is being invalidate because project ea4da625d0f34b628068fd1300d971ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81617) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:57:21.740365 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-e2b0efda-e3e1-4984-85f5-d489984be518 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:21.740986 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-e2b0efda-e3e1-4984-85f5-d489984be518 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:21.742493 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1447/2883] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:57:21 2022] DELETE /identity/v3/projects/ea4da625d0f34b628068fd1300d971ab => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:57:21.746611 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c65ae2-9699-4b91-ae15-28beea12140c service neutron] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:21.756257 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c65ae2-9699-4b91-ae15-28beea12140c service neutron] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:21.812598 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bff9030-a46b-47bc-ac74-ff47d745043f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:21.813214 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bff9030-a46b-47bc-ac74-ff47d745043f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:21.813491 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bff9030-a46b-47bc-ac74-ff47d745043f admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:21.813757 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bff9030-a46b-47bc-ac74-ff47d745043f admin admin] PATH_INFO: `/v3/users/cd946315d76b473e8e098530b8fe22d8` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:21.814885 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bff9030-a46b-47bc-ac74-ff47d745043f admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd946315d76b473e8e098530b8fe22d8)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:21.816760 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bff9030-a46b-47bc-ac74-ff47d745043f admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:21.861467 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-2bff9030-a46b-47bc-ac74-ff47d745043f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd946315d76b473e8e098530b8fe22d8'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:21.868500 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-2bff9030-a46b-47bc-ac74-ff47d745043f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd946315d76b473e8e098530b8fe22d8'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:21.881476 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-2bff9030-a46b-47bc-ac74-ff47d745043f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd946315d76b473e8e098530b8fe22d8'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:21.888784 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-2bff9030-a46b-47bc-ac74-ff47d745043f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd946315d76b473e8e098530b8fe22d8'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:21.889447 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-2bff9030-a46b-47bc-ac74-ff47d745043f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd946315d76b473e8e098530b8fe22d8'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:21.896233 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1437/2884] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:57:21 2022] DELETE /identity/v3/users/cd946315d76b473e8e098530b8fe22d8 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:57:22.180876 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b0efda-e3e1-4984-85f5-d489984be518 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:22.182449 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-e2b0efda-e3e1-4984-85f5-d489984be518 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:22.196806 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b0efda-e3e1-4984-85f5-d489984be518 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:22.257479 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df311e3e-b2ce-4282-b551-51a4e09469e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:22.258031 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df311e3e-b2ce-4282-b551-51a4e09469e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:22.258313 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df311e3e-b2ce-4282-b551-51a4e09469e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:22.258618 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df311e3e-b2ce-4282-b551-51a4e09469e5 admin admin] PATH_INFO: `/v3/projects/b0cc985c9eed4295975b16bef69fd914` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:22.259744 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df311e3e-b2ce-4282-b551-51a4e09469e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0cc985c9eed4295975b16bef69fd914)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:22.261649 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df311e3e-b2ce-4282-b551-51a4e09469e5 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:22.308274 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-df311e3e-b2ce-4282-b551-51a4e09469e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0cc985c9eed4295975b16bef69fd914', 'request_id': 'req-df311e3e-b2ce-4282-b551-51a4e09469e5'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:22.315008 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-df311e3e-b2ce-4282-b551-51a4e09469e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0cc985c9eed4295975b16bef69fd914', 'request_id': 'req-df311e3e-b2ce-4282-b551-51a4e09469e5'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:22.320611 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-df311e3e-b2ce-4282-b551-51a4e09469e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0cc985c9eed4295975b16bef69fd914', 'request_id': 'req-df311e3e-b2ce-4282-b551-51a4e09469e5'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:22.334034 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-df311e3e-b2ce-4282-b551-51a4e09469e5 admin admin] The token cache is being invalidate because project b0cc985c9eed4295975b16bef69fd914 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81617) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:57:22.334326 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-df311e3e-b2ce-4282-b551-51a4e09469e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:22.334978 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-df311e3e-b2ce-4282-b551-51a4e09469e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:22.336461 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1448/2885] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:57:22 2022] DELETE /identity/v3/projects/b0cc985c9eed4295975b16bef69fd914 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:57:22.341482 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a595645-92d2-44c6-94a5-2772d100ff67 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:22.342000 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a595645-92d2-44c6-94a5-2772d100ff67 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:22.342000 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a595645-92d2-44c6-94a5-2772d100ff67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:22.362423 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-1a595645-92d2-44c6-94a5-2772d100ff67 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:57:22.363642 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-1a595645-92d2-44c6-94a5-2772d100ff67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:22.436215 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1438/2886] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:57:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:57:22.449562 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e1837-2867-4d50-85f1-aba91e539ff1 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:22.449845 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e1837-2867-4d50-85f1-aba91e539ff1 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:22.450122 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e1837-2867-4d50-85f1-aba91e539ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:22.470045 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-6c4e1837-2867-4d50-85f1-aba91e539ff1 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:57:22.471298 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-6c4e1837-2867-4d50-85f1-aba91e539ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:22.479089 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1449/2887] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:57:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:57:22.483326 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a595645-92d2-44c6-94a5-2772d100ff67 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:22.493852 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a595645-92d2-44c6-94a5-2772d100ff67 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:22.498461 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbe16e5-32b1-4eec-b0fd-29efce0f1473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:22.499117 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe16e5-32b1-4eec-b0fd-29efce0f1473 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:22.499483 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe16e5-32b1-4eec-b0fd-29efce0f1473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:22.499858 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe16e5-32b1-4eec-b0fd-29efce0f1473 admin admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:22.500415 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbe16e5-32b1-4eec-b0fd-29efce0f1473 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:57:22.500719 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbe16e5-32b1-4eec-b0fd-29efce0f1473 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:22.502290 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbe16e5-32b1-4eec-b0fd-29efce0f1473 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:22.509063 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:57:22.515109 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1439/2888] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:57:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 16:57:22.519557 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4e1837-2867-4d50-85f1-aba91e539ff1 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:22.528906 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4e1837-2867-4d50-85f1-aba91e539ff1 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:22.534004 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed99b27-92a2-4b1e-a89c-56eba11e8d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:22.534630 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed99b27-92a2-4b1e-a89c-56eba11e8d15 admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:22.534912 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed99b27-92a2-4b1e-a89c-56eba11e8d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:22.535185 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed99b27-92a2-4b1e-a89c-56eba11e8d15 admin admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:22.535824 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed99b27-92a2-4b1e-a89c-56eba11e8d15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:22.537714 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed99b27-92a2-4b1e-a89c-56eba11e8d15 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:22.564710 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1450/2889] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:57:22 2022] POST /identity/v3/projects => generated 357 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:57:22.568955 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbe16e5-32b1-4eec-b0fd-29efce0f1473 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:22.579433 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbe16e5-32b1-4eec-b0fd-29efce0f1473 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:22.641820 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e738ef5-a1e9-455b-99e4-0d581de80676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:22.642502 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e738ef5-a1e9-455b-99e4-0d581de80676 admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:22.642778 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e738ef5-a1e9-455b-99e4-0d581de80676 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:22.643050 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e738ef5-a1e9-455b-99e4-0d581de80676 admin admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:22.643679 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e738ef5-a1e9-455b-99e4-0d581de80676 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:22.645590 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e738ef5-a1e9-455b-99e4-0d581de80676 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:22.666489 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1440/2890] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:57:22 2022] POST /identity/v3/users => generated 279 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:57:22.670768 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed99b27-92a2-4b1e-a89c-56eba11e8d15 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:22.681073 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed99b27-92a2-4b1e-a89c-56eba11e8d15 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:22.685923 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcf7e4b-47d9-4f53-adab-e49722ace6ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:22.686637 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf7e4b-47d9-4f53-adab-e49722ace6ff admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:22.686920 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf7e4b-47d9-4f53-adab-e49722ace6ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:22.687193 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf7e4b-47d9-4f53-adab-e49722ace6ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:22.687832 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcf7e4b-47d9-4f53-adab-e49722ace6ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:22.689412 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcf7e4b-47d9-4f53-adab-e49722ace6ff admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:22.691463 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:57:22.703492 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1451/2891] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:57:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:57:22.704662 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e738ef5-a1e9-455b-99e4-0d581de80676 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:22.716575 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e738ef5-a1e9-455b-99e4-0d581de80676 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:22.722457 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa9fc3d-e9d8-434b-bc79-046842b5a942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:22.724886 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa9fc3d-e9d8-434b-bc79-046842b5a942 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:22.724886 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa9fc3d-e9d8-434b-bc79-046842b5a942 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:22.724886 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa9fc3d-e9d8-434b-bc79-046842b5a942 admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:22.724886 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa9fc3d-e9d8-434b-bc79-046842b5a942 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:22.725973 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa9fc3d-e9d8-434b-bc79-046842b5a942 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:22.738092 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1441/2892] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:57:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:57:22.742419 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcf7e4b-47d9-4f53-adab-e49722ace6ff admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:22.752432 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcf7e4b-47d9-4f53-adab-e49722ace6ff admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:22.757062 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fad4c1-8d07-4f4d-8f36-d26e1ae3e69b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:22.757707 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fad4c1-8d07-4f4d-8f36-d26e1ae3e69b admin admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:22.757982 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fad4c1-8d07-4f4d-8f36-d26e1ae3e69b admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:22.758257 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fad4c1-8d07-4f4d-8f36-d26e1ae3e69b admin admin] PATH_INFO: `/v3/projects/a04052a082a6451b988aaf1f75c149b6/users/6cc4a1a7240c4cbe829edc2d9b06b086/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:22.774248 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fad4c1-8d07-4f4d-8f36-d26e1ae3e69b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a04052a082a6451b988aaf1f75c149b6, user_id=6cc4a1a7240c4cbe829edc2d9b06b086, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:22.776853 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fad4c1-8d07-4f4d-8f36-d26e1ae3e69b admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:22.792631 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1452/2893] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:57:22 2022] PUT /identity/v3/projects/a04052a082a6451b988aaf1f75c149b6/users/6cc4a1a7240c4cbe829edc2d9b06b086/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:57:22.802529 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e6583-492f-42b2-b7c0-597ba9ccd931 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:22.802811 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e6583-492f-42b2-b7c0-597ba9ccd931 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:22.803081 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e6583-492f-42b2-b7c0-597ba9ccd931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:22.830910 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-b61e6583-492f-42b2-b7c0-597ba9ccd931 None None] MFA Rules not processed for user `6cc4a1a7240c4cbe829edc2d9b06b086`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:57:22.832126 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-b61e6583-492f-42b2-b7c0-597ba9ccd931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:22.895628 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1442/2894] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:57:22 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:57:22.900117 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fad4c1-8d07-4f4d-8f36-d26e1ae3e69b admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:22.963227 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fad4c1-8d07-4f4d-8f36-d26e1ae3e69b admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:22.968131 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4e0da5-c4ac-4db4-92e0-de5ed84d7602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:22.968845 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4e0da5-c4ac-4db4-92e0-de5ed84d7602 admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:22.969240 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4e0da5-c4ac-4db4-92e0-de5ed84d7602 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:22.969654 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4e0da5-c4ac-4db4-92e0-de5ed84d7602 admin admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:22.970331 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4e0da5-c4ac-4db4-92e0-de5ed84d7602 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:22.972348 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4e0da5-c4ac-4db4-92e0-de5ed84d7602 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:23.002016 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1453/2895] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:57:22 2022] POST /identity/v3/projects => generated 355 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 16:57:23.003354 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61e6583-492f-42b2-b7c0-597ba9ccd931 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:23.013873 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61e6583-492f-42b2-b7c0-597ba9ccd931 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:23.070781 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ec2c43-e39c-4e91-a720-87859d2b9b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:23.071310 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec2c43-e39c-4e91-a720-87859d2b9b62 admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:23.071591 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec2c43-e39c-4e91-a720-87859d2b9b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:23.071865 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec2c43-e39c-4e91-a720-87859d2b9b62 admin admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:23.072532 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ec2c43-e39c-4e91-a720-87859d2b9b62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:23.074493 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ec2c43-e39c-4e91-a720-87859d2b9b62 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:23.086769 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:57:23.102610 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1443/2896] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:57:23 2022] POST /identity/v3/users => generated 284 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:57:23.103622 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4e0da5-c4ac-4db4-92e0-de5ed84d7602 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:23.114897 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4e0da5-c4ac-4db4-92e0-de5ed84d7602 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:23.119537 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db90739-d3da-4673-812b-c6f835656817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:23.120133 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db90739-d3da-4673-812b-c6f835656817 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:23.120408 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db90739-d3da-4673-812b-c6f835656817 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:23.120676 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db90739-d3da-4673-812b-c6f835656817 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:23.121310 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db90739-d3da-4673-812b-c6f835656817 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:23.122935 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db90739-d3da-4673-812b-c6f835656817 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:23.135202 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1454/2897] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:57:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:57:23.139458 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ec2c43-e39c-4e91-a720-87859d2b9b62 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:23.150236 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ec2c43-e39c-4e91-a720-87859d2b9b62 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:23.154983 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5903cde-153f-42b3-9bdc-dcb8fd26d391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:23.155726 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5903cde-153f-42b3-9bdc-dcb8fd26d391 admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:23.156003 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5903cde-153f-42b3-9bdc-dcb8fd26d391 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:23.156278 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5903cde-153f-42b3-9bdc-dcb8fd26d391 admin admin] PATH_INFO: `/v3/projects/2c4e868c03394225a4f270157fc2fff6/users/7c1f1f48c3124210b21e51c391e7a4d5/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:23.171619 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5903cde-153f-42b3-9bdc-dcb8fd26d391 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c4e868c03394225a4f270157fc2fff6, user_id=7c1f1f48c3124210b21e51c391e7a4d5, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:23.174241 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5903cde-153f-42b3-9bdc-dcb8fd26d391 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:23.188691 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1444/2898] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:57:23 2022] PUT /identity/v3/projects/2c4e868c03394225a4f270157fc2fff6/users/7c1f1f48c3124210b21e51c391e7a4d5/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:57:23.192813 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db90739-d3da-4673-812b-c6f835656817 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:23.204914 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db90739-d3da-4673-812b-c6f835656817 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:23.263069 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62035dd3-54be-4ee2-af6c-f732ed120f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:23.263678 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62035dd3-54be-4ee2-af6c-f732ed120f75 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:23.263958 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62035dd3-54be-4ee2-af6c-f732ed120f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:23.264233 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62035dd3-54be-4ee2-af6c-f732ed120f75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:23.265434 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62035dd3-54be-4ee2-af6c-f732ed120f75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:23.267069 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62035dd3-54be-4ee2-af6c-f732ed120f75 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:23.277115 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1455/2899] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:57:23 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:57:23.281606 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5903cde-153f-42b3-9bdc-dcb8fd26d391 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:23.292109 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5903cde-153f-42b3-9bdc-dcb8fd26d391 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:23.296671 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48f3ccb-a6f2-4d79-a5cb-dcdc55f17054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:23.297322 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f3ccb-a6f2-4d79-a5cb-dcdc55f17054 admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:23.297598 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f3ccb-a6f2-4d79-a5cb-dcdc55f17054 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:23.297867 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f3ccb-a6f2-4d79-a5cb-dcdc55f17054 admin admin] PATH_INFO: `/v3/projects/2c4e868c03394225a4f270157fc2fff6/users/7c1f1f48c3124210b21e51c391e7a4d5/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:23.299784 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48f3ccb-a6f2-4d79-a5cb-dcdc55f17054 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c4e868c03394225a4f270157fc2fff6, user_id=7c1f1f48c3124210b21e51c391e7a4d5, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:23.302403 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48f3ccb-a6f2-4d79-a5cb-dcdc55f17054 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:23.313860 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1445/2900] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:57:23 2022] PUT /identity/v3/projects/2c4e868c03394225a4f270157fc2fff6/users/7c1f1f48c3124210b21e51c391e7a4d5/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:57:23.324382 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e0460e-c01e-4a26-b060-791ee3000ccc None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:23.324591 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e0460e-c01e-4a26-b060-791ee3000ccc None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:23.324873 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e0460e-c01e-4a26-b060-791ee3000ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:23.354240 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-62e0460e-c01e-4a26-b060-791ee3000ccc None None] MFA Rules not processed for user `7c1f1f48c3124210b21e51c391e7a4d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:57:23.355721 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-62e0460e-c01e-4a26-b060-791ee3000ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:23.391533 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:57:23.428107 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1456/2901] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:57:23 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:57:23.441467 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48f3ccb-a6f2-4d79-a5cb-dcdc55f17054 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:23.443388 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-f48f3ccb-a6f2-4d79-a5cb-dcdc55f17054 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:23.500952 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48f3ccb-a6f2-4d79-a5cb-dcdc55f17054 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:23.504631 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030cf5ee-4cc5-4328-8af9-aac0c960fea0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:23.505229 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030cf5ee-4cc5-4328-8af9-aac0c960fea0 service cinder] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:23.505514 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030cf5ee-4cc5-4328-8af9-aac0c960fea0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:23.505786 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030cf5ee-4cc5-4328-8af9-aac0c960fea0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:23.514874 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030cf5ee-4cc5-4328-8af9-aac0c960fea0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:23.516337 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030cf5ee-4cc5-4328-8af9-aac0c960fea0 service cinder] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:23.525816 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1446/2902] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:57:23 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:57:23.603710 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e0460e-c01e-4a26-b060-791ee3000ccc None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:23.613937 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e0460e-c01e-4a26-b060-791ee3000ccc None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:23.617535 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a8460c-6923-4f61-8f72-7bd75690ac31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:23.618119 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a8460c-6923-4f61-8f72-7bd75690ac31 service cinder] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:23.618417 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a8460c-6923-4f61-8f72-7bd75690ac31 service cinder] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:23.618689 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a8460c-6923-4f61-8f72-7bd75690ac31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:23.676250 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a8460c-6923-4f61-8f72-7bd75690ac31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:23.677694 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a8460c-6923-4f61-8f72-7bd75690ac31 service cinder] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:23.687573 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1457/2903] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:57:23 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:57:33.521704 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=43,keystone:INSERT=2 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:57:33.682546 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=42 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:57:38.752253 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030cf5ee-4cc5-4328-8af9-aac0c960fea0 service cinder] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:38.814748 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030cf5ee-4cc5-4328-8af9-aac0c960fea0 service cinder] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:38.819265 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2c349f-44f6-404b-93f6-edb714c0789d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:38.819887 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c349f-44f6-404b-93f6-edb714c0789d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:38.820167 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c349f-44f6-404b-93f6-edb714c0789d admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:38.820442 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c349f-44f6-404b-93f6-edb714c0789d admin admin] PATH_INFO: `/v3/users/6cc4a1a7240c4cbe829edc2d9b06b086` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:38.821550 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2c349f-44f6-404b-93f6-edb714c0789d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cc4a1a7240c4cbe829edc2d9b06b086)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:38.823483 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2c349f-44f6-404b-93f6-edb714c0789d admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:38.869982 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-4a2c349f-44f6-404b-93f6-edb714c0789d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cc4a1a7240c4cbe829edc2d9b06b086'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:38.877419 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-4a2c349f-44f6-404b-93f6-edb714c0789d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cc4a1a7240c4cbe829edc2d9b06b086'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:38.889253 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-4a2c349f-44f6-404b-93f6-edb714c0789d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cc4a1a7240c4cbe829edc2d9b06b086'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:38.896636 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-4a2c349f-44f6-404b-93f6-edb714c0789d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cc4a1a7240c4cbe829edc2d9b06b086'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:38.897262 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-4a2c349f-44f6-404b-93f6-edb714c0789d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cc4a1a7240c4cbe829edc2d9b06b086'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:38.903228 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1447/2904] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:57:38 2022] DELETE /identity/v3/users/6cc4a1a7240c4cbe829edc2d9b06b086 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:57:38.908213 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee95177-63e0-4ec7-b758-903042e38b78 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:38.908708 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee95177-63e0-4ec7-b758-903042e38b78 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:38.908792 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee95177-63e0-4ec7-b758-903042e38b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:38.928139 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-8ee95177-63e0-4ec7-b758-903042e38b78 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:57:38.929338 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-8ee95177-63e0-4ec7-b758-903042e38b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:38.987628 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1458/2905] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:57:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:57:38.997497 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2c349f-44f6-404b-93f6-edb714c0789d admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:38.999317 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-4a2c349f-44f6-404b-93f6-edb714c0789d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:39.062825 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2c349f-44f6-404b-93f6-edb714c0789d admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:39.066482 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffb1f82-7428-4a59-97f9-dc3fbb65c071 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:39.067085 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffb1f82-7428-4a59-97f9-dc3fbb65c071 service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:39.067363 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffb1f82-7428-4a59-97f9-dc3fbb65c071 service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:39.067633 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffb1f82-7428-4a59-97f9-dc3fbb65c071 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:39.082187 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffb1f82-7428-4a59-97f9-dc3fbb65c071 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:39.083648 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffb1f82-7428-4a59-97f9-dc3fbb65c071 service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:39.094047 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1448/2906] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:57:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:57:39.355826 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee95177-63e0-4ec7-b758-903042e38b78 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:39.357299 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-8ee95177-63e0-4ec7-b758-903042e38b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:39.369821 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee95177-63e0-4ec7-b758-903042e38b78 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:39.374429 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068cb390-df4e-4c55-a3ab-cccb29b328c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:39.375065 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068cb390-df4e-4c55-a3ab-cccb29b328c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:39.375345 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068cb390-df4e-4c55-a3ab-cccb29b328c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:39.375622 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068cb390-df4e-4c55-a3ab-cccb29b328c0 admin admin] PATH_INFO: `/v3/projects/a04052a082a6451b988aaf1f75c149b6` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:39.376754 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068cb390-df4e-4c55-a3ab-cccb29b328c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a04052a082a6451b988aaf1f75c149b6)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:39.378695 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068cb390-df4e-4c55-a3ab-cccb29b328c0 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:39.426857 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-068cb390-df4e-4c55-a3ab-cccb29b328c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a04052a082a6451b988aaf1f75c149b6', 'request_id': 'req-068cb390-df4e-4c55-a3ab-cccb29b328c0'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:39.432896 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-068cb390-df4e-4c55-a3ab-cccb29b328c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a04052a082a6451b988aaf1f75c149b6', 'request_id': 'req-068cb390-df4e-4c55-a3ab-cccb29b328c0'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:39.439239 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-068cb390-df4e-4c55-a3ab-cccb29b328c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a04052a082a6451b988aaf1f75c149b6', 'request_id': 'req-068cb390-df4e-4c55-a3ab-cccb29b328c0'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:39.453000 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-068cb390-df4e-4c55-a3ab-cccb29b328c0 admin admin] The token cache is being invalidate because project a04052a082a6451b988aaf1f75c149b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81617) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:57:39.453292 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-068cb390-df4e-4c55-a3ab-cccb29b328c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:39.453910 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-068cb390-df4e-4c55-a3ab-cccb29b328c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:39.455447 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1459/2907] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:57:39 2022] DELETE /identity/v3/projects/a04052a082a6451b988aaf1f75c149b6 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:57:39.459384 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffb1f82-7428-4a59-97f9-dc3fbb65c071 service neutron] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:39.468875 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffb1f82-7428-4a59-97f9-dc3fbb65c071 service neutron] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:39.487156 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:57:39.527670 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96293348-a844-4591-be6b-bc8ee46c68f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:39.528216 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96293348-a844-4591-be6b-bc8ee46c68f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:39.528499 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96293348-a844-4591-be6b-bc8ee46c68f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:39.528775 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96293348-a844-4591-be6b-bc8ee46c68f1 admin admin] PATH_INFO: `/v3/users/7c1f1f48c3124210b21e51c391e7a4d5` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:39.529920 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96293348-a844-4591-be6b-bc8ee46c68f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c1f1f48c3124210b21e51c391e7a4d5)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:39.531888 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96293348-a844-4591-be6b-bc8ee46c68f1 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:39.578122 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-96293348-a844-4591-be6b-bc8ee46c68f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c1f1f48c3124210b21e51c391e7a4d5'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:39.584923 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-96293348-a844-4591-be6b-bc8ee46c68f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c1f1f48c3124210b21e51c391e7a4d5'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:39.597217 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-96293348-a844-4591-be6b-bc8ee46c68f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c1f1f48c3124210b21e51c391e7a4d5'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:39.605217 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-96293348-a844-4591-be6b-bc8ee46c68f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c1f1f48c3124210b21e51c391e7a4d5'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:39.605878 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-96293348-a844-4591-be6b-bc8ee46c68f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c1f1f48c3124210b21e51c391e7a4d5'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:39.612384 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1449/2908] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:57:39 2022] DELETE /identity/v3/users/7c1f1f48c3124210b21e51c391e7a4d5 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:57:39.883294 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068cb390-df4e-4c55-a3ab-cccb29b328c0 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:39.884754 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-068cb390-df4e-4c55-a3ab-cccb29b328c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:39.899687 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068cb390-df4e-4c55-a3ab-cccb29b328c0 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:39.963368 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a94924-2f17-4bee-9ce5-318699921c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:39.964445 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a94924-2f17-4bee-9ce5-318699921c73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:39.964445 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a94924-2f17-4bee-9ce5-318699921c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:39.964445 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a94924-2f17-4bee-9ce5-318699921c73 admin admin] PATH_INFO: `/v3/projects/2c4e868c03394225a4f270157fc2fff6` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:39.965573 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a94924-2f17-4bee-9ce5-318699921c73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c4e868c03394225a4f270157fc2fff6)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:39.968085 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a94924-2f17-4bee-9ce5-318699921c73 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:39.999171 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:57:40.016803 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-a1a94924-2f17-4bee-9ce5-318699921c73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c4e868c03394225a4f270157fc2fff6', 'request_id': 'req-a1a94924-2f17-4bee-9ce5-318699921c73'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:40.022789 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-a1a94924-2f17-4bee-9ce5-318699921c73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c4e868c03394225a4f270157fc2fff6', 'request_id': 'req-a1a94924-2f17-4bee-9ce5-318699921c73'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:40.028154 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-a1a94924-2f17-4bee-9ce5-318699921c73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c4e868c03394225a4f270157fc2fff6', 'request_id': 'req-a1a94924-2f17-4bee-9ce5-318699921c73'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:40.041250 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-a1a94924-2f17-4bee-9ce5-318699921c73 admin admin] The token cache is being invalidate because project 2c4e868c03394225a4f270157fc2fff6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81617) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:57:40.041475 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-a1a94924-2f17-4bee-9ce5-318699921c73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:40.042092 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-a1a94924-2f17-4bee-9ce5-318699921c73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:40.043706 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1460/2909] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:57:39 2022] DELETE /identity/v3/projects/2c4e868c03394225a4f270157fc2fff6 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:57:40.049335 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d908709d-8b6d-4e0d-92c3-4523050dbd94 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:40.049803 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d908709d-8b6d-4e0d-92c3-4523050dbd94 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:40.049917 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d908709d-8b6d-4e0d-92c3-4523050dbd94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:40.069752 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-d908709d-8b6d-4e0d-92c3-4523050dbd94 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:57:40.071017 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-d908709d-8b6d-4e0d-92c3-4523050dbd94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:40.132155 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1450/2910] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:57:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:57:40.140418 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5130aa1f-6cc8-4f31-ab9d-3670a5f64cc5 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:40.140707 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5130aa1f-6cc8-4f31-ab9d-3670a5f64cc5 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:40.140984 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5130aa1f-6cc8-4f31-ab9d-3670a5f64cc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:40.161354 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-5130aa1f-6cc8-4f31-ab9d-3670a5f64cc5 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:57:40.162621 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-5130aa1f-6cc8-4f31-ab9d-3670a5f64cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:40.170749 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1461/2911] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:57:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:57:40.176158 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d908709d-8b6d-4e0d-92c3-4523050dbd94 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:40.190855 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d908709d-8b6d-4e0d-92c3-4523050dbd94 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:40.195460 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d65b939-e519-4e70-bd26-af194e7a55a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:40.196059 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d65b939-e519-4e70-bd26-af194e7a55a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:40.196341 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d65b939-e519-4e70-bd26-af194e7a55a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:40.196617 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d65b939-e519-4e70-bd26-af194e7a55a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:40.197336 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d65b939-e519-4e70-bd26-af194e7a55a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:57:40.197652 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d65b939-e519-4e70-bd26-af194e7a55a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:40.199287 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d65b939-e519-4e70-bd26-af194e7a55a6 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:40.211453 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1451/2912] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:57:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:57:40.215784 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5130aa1f-6cc8-4f31-ab9d-3670a5f64cc5 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:40.225092 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5130aa1f-6cc8-4f31-ab9d-3670a5f64cc5 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:40.229748 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b8cea4-0d60-4940-b675-8b0208d3e1bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:40.230344 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b8cea4-0d60-4940-b675-8b0208d3e1bd admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:40.230662 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b8cea4-0d60-4940-b675-8b0208d3e1bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:40.230933 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b8cea4-0d60-4940-b675-8b0208d3e1bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:40.231576 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b8cea4-0d60-4940-b675-8b0208d3e1bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:40.233532 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b8cea4-0d60-4940-b675-8b0208d3e1bd admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:40.265459 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1462/2913] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:57:40 2022] POST /identity/v3/projects => generated 387 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:57:40.270478 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d65b939-e519-4e70-bd26-af194e7a55a6 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:40.281141 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d65b939-e519-4e70-bd26-af194e7a55a6 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:40.314029 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:57:40.350956 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f04952-2ba2-4a78-a7b4-fe19d0e4982f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:40.351577 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f04952-2ba2-4a78-a7b4-fe19d0e4982f admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:40.351862 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f04952-2ba2-4a78-a7b4-fe19d0e4982f admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:40.352138 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f04952-2ba2-4a78-a7b4-fe19d0e4982f admin admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:40.352791 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f04952-2ba2-4a78-a7b4-fe19d0e4982f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:40.354718 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f04952-2ba2-4a78-a7b4-fe19d0e4982f admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:40.377243 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1452/2914] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:57:40 2022] POST /identity/v3/users => generated 294 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:57:40.381749 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b8cea4-0d60-4940-b675-8b0208d3e1bd admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:40.392985 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b8cea4-0d60-4940-b675-8b0208d3e1bd admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:40.397607 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021a71d4-995b-468b-876c-0541e412e58c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:40.398205 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021a71d4-995b-468b-876c-0541e412e58c admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:40.398525 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021a71d4-995b-468b-876c-0541e412e58c admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:40.398800 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021a71d4-995b-468b-876c-0541e412e58c admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:40.399441 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021a71d4-995b-468b-876c-0541e412e58c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:40.401017 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021a71d4-995b-468b-876c-0541e412e58c admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:40.410860 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1463/2915] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:57:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:57:40.414828 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f04952-2ba2-4a78-a7b4-fe19d0e4982f admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:40.424761 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f04952-2ba2-4a78-a7b4-fe19d0e4982f admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:40.429958 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edb688f-ec3d-476b-8225-69fa238c72b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:40.430592 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb688f-ec3d-476b-8225-69fa238c72b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:40.430871 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb688f-ec3d-476b-8225-69fa238c72b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:40.431145 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb688f-ec3d-476b-8225-69fa238c72b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:40.431814 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edb688f-ec3d-476b-8225-69fa238c72b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:40.433387 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edb688f-ec3d-476b-8225-69fa238c72b1 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:40.444055 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1453/2916] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:57:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:57:40.448436 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021a71d4-995b-468b-876c-0541e412e58c admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:40.458925 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021a71d4-995b-468b-876c-0541e412e58c admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:40.463605 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89889c27-086b-42f1-8787-15ab1478262b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:40.464250 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89889c27-086b-42f1-8787-15ab1478262b admin admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:40.464529 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89889c27-086b-42f1-8787-15ab1478262b admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:40.464803 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89889c27-086b-42f1-8787-15ab1478262b admin admin] PATH_INFO: `/v3/projects/552d8c6a623645258615e961b076e5c5/users/4290c601aa144595a6f7819d17faf339/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:40.479284 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89889c27-086b-42f1-8787-15ab1478262b admin admin] RBAC: Authorizing `identity:create_grant(project_id=552d8c6a623645258615e961b076e5c5, user_id=4290c601aa144595a6f7819d17faf339, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:40.481919 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89889c27-086b-42f1-8787-15ab1478262b admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:40.494706 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1464/2917] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:57:40 2022] PUT /identity/v3/projects/552d8c6a623645258615e961b076e5c5/users/4290c601aa144595a6f7819d17faf339/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:57:40.505640 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c528027-485f-4d13-85fa-e7b4164681a6 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:40.505923 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c528027-485f-4d13-85fa-e7b4164681a6 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:40.506197 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c528027-485f-4d13-85fa-e7b4164681a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:40.535122 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-5c528027-485f-4d13-85fa-e7b4164681a6 None None] MFA Rules not processed for user `4290c601aa144595a6f7819d17faf339`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:57:40.536345 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-5c528027-485f-4d13-85fa-e7b4164681a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:40.612140 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1454/2918] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:57:40 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:57:40.616727 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89889c27-086b-42f1-8787-15ab1478262b admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:40.679867 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89889c27-086b-42f1-8787-15ab1478262b admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:40.685939 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bf0f94-a804-4f90-bb96-a9104cbb9b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:40.687114 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bf0f94-a804-4f90-bb96-a9104cbb9b5e admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:40.687421 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bf0f94-a804-4f90-bb96-a9104cbb9b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:40.687670 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bf0f94-a804-4f90-bb96-a9104cbb9b5e admin admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:40.688319 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bf0f94-a804-4f90-bb96-a9104cbb9b5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:40.690229 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bf0f94-a804-4f90-bb96-a9104cbb9b5e admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:40.720780 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1465/2919] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:57:40 2022] POST /identity/v3/projects => generated 387 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:57:40.726178 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c528027-485f-4d13-85fa-e7b4164681a6 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:40.738263 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c528027-485f-4d13-85fa-e7b4164681a6 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:40.796078 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8571e98-bf69-4cba-883f-9bd84f2dd0d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:40.796741 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8571e98-bf69-4cba-883f-9bd84f2dd0d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:40.796964 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8571e98-bf69-4cba-883f-9bd84f2dd0d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:40.797271 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8571e98-bf69-4cba-883f-9bd84f2dd0d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:40.797992 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8571e98-bf69-4cba-883f-9bd84f2dd0d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:40.800276 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8571e98-bf69-4cba-883f-9bd84f2dd0d6 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:40.823374 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1455/2920] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:57:40 2022] POST /identity/v3/users => generated 300 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:57:40.827437 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bf0f94-a804-4f90-bb96-a9104cbb9b5e admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:40.837701 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bf0f94-a804-4f90-bb96-a9104cbb9b5e admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:40.842305 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfca550f-72c1-4f1d-90cd-36933230c910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:40.842936 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfca550f-72c1-4f1d-90cd-36933230c910 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:40.843211 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfca550f-72c1-4f1d-90cd-36933230c910 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:40.843481 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfca550f-72c1-4f1d-90cd-36933230c910 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:40.844119 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfca550f-72c1-4f1d-90cd-36933230c910 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:40.845741 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfca550f-72c1-4f1d-90cd-36933230c910 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:40.857277 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1466/2921] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:57:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 16:57:40.862642 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8571e98-bf69-4cba-883f-9bd84f2dd0d6 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:40.872746 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8571e98-bf69-4cba-883f-9bd84f2dd0d6 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:40.878933 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e75805-0e78-494e-916e-363e4d81bb02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:40.879599 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e75805-0e78-494e-916e-363e4d81bb02 admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:40.879881 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e75805-0e78-494e-916e-363e4d81bb02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:40.880163 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e75805-0e78-494e-916e-363e4d81bb02 admin admin] PATH_INFO: `/v3/projects/f4869a6405554338a6fc246699bde0ff/users/062861f8618c4fbdbee4d4bda7783481/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:40.898472 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e75805-0e78-494e-916e-363e4d81bb02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4869a6405554338a6fc246699bde0ff, user_id=062861f8618c4fbdbee4d4bda7783481, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:40.902945 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e75805-0e78-494e-916e-363e4d81bb02 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:40.921493 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1456/2922] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:57:40 2022] PUT /identity/v3/projects/f4869a6405554338a6fc246699bde0ff/users/062861f8618c4fbdbee4d4bda7783481/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:57:40.926646 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfca550f-72c1-4f1d-90cd-36933230c910 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:40.940500 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfca550f-72c1-4f1d-90cd-36933230c910 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:40.962062 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:57:41.002279 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00e82cc-a0a0-4c19-883f-436b2b1efc48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:41.002978 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00e82cc-a0a0-4c19-883f-436b2b1efc48 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:41.003202 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00e82cc-a0a0-4c19-883f-436b2b1efc48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:41.003476 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00e82cc-a0a0-4c19-883f-436b2b1efc48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:41.004118 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00e82cc-a0a0-4c19-883f-436b2b1efc48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:41.005710 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00e82cc-a0a0-4c19-883f-436b2b1efc48 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:41.015812 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1467/2923] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:57:40 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:57:41.021232 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e75805-0e78-494e-916e-363e4d81bb02 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:41.026644 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:57:41.033878 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e75805-0e78-494e-916e-363e4d81bb02 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:41.038704 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba8b43c-998c-4ceb-99fe-7f576d1d9b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:41.039359 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba8b43c-998c-4ceb-99fe-7f576d1d9b56 admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:41.039643 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba8b43c-998c-4ceb-99fe-7f576d1d9b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:41.039919 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba8b43c-998c-4ceb-99fe-7f576d1d9b56 admin admin] PATH_INFO: `/v3/projects/f4869a6405554338a6fc246699bde0ff/users/062861f8618c4fbdbee4d4bda7783481/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:41.041838 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba8b43c-998c-4ceb-99fe-7f576d1d9b56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4869a6405554338a6fc246699bde0ff, user_id=062861f8618c4fbdbee4d4bda7783481, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:41.044523 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba8b43c-998c-4ceb-99fe-7f576d1d9b56 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:41.055949 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1457/2924] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:57:41 2022] PUT /identity/v3/projects/f4869a6405554338a6fc246699bde0ff/users/062861f8618c4fbdbee4d4bda7783481/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:57:41.068981 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484460ec-a54b-43df-a48f-91740d5eb9c7 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:41.069263 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484460ec-a54b-43df-a48f-91740d5eb9c7 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:41.069542 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484460ec-a54b-43df-a48f-91740d5eb9c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:41.098655 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-484460ec-a54b-43df-a48f-91740d5eb9c7 None None] MFA Rules not processed for user `062861f8618c4fbdbee4d4bda7783481`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:57:41.099923 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-484460ec-a54b-43df-a48f-91740d5eb9c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:41.168922 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1468/2925] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:57:41 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:57:41.181177 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba8b43c-998c-4ceb-99fe-7f576d1d9b56 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:41.184008 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-5ba8b43c-998c-4ceb-99fe-7f576d1d9b56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:41.245189 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba8b43c-998c-4ceb-99fe-7f576d1d9b56 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:41.248890 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdabac8-fab7-420f-be85-c4d67353bd4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:41.249495 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdabac8-fab7-420f-be85-c4d67353bd4b service cinder] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:41.249770 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdabac8-fab7-420f-be85-c4d67353bd4b service cinder] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:41.250040 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdabac8-fab7-420f-be85-c4d67353bd4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:41.259582 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdabac8-fab7-420f-be85-c4d67353bd4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:41.260966 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdabac8-fab7-420f-be85-c4d67353bd4b service cinder] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:41.270999 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1458/2926] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:57:41 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:57:41.300958 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484460ec-a54b-43df-a48f-91740d5eb9c7 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:41.376140 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484460ec-a54b-43df-a48f-91740d5eb9c7 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:41.380750 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17ebe92-471e-42df-9ad4-b5cf2ba5c944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:41.381384 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17ebe92-471e-42df-9ad4-b5cf2ba5c944 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:41.381672 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17ebe92-471e-42df-9ad4-b5cf2ba5c944 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:41.381939 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17ebe92-471e-42df-9ad4-b5cf2ba5c944 admin admin] PATH_INFO: `/v3/users/4290c601aa144595a6f7819d17faf339` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:41.383096 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17ebe92-471e-42df-9ad4-b5cf2ba5c944 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4290c601aa144595a6f7819d17faf339)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:41.385034 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17ebe92-471e-42df-9ad4-b5cf2ba5c944 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:41.435860 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-d17ebe92-471e-42df-9ad4-b5cf2ba5c944 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4290c601aa144595a6f7819d17faf339'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:41.436460 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-d17ebe92-471e-42df-9ad4-b5cf2ba5c944 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4290c601aa144595a6f7819d17faf339'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:41.437759 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:57:41.444887 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-d17ebe92-471e-42df-9ad4-b5cf2ba5c944 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4290c601aa144595a6f7819d17faf339'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:41.460358 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-d17ebe92-471e-42df-9ad4-b5cf2ba5c944 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4290c601aa144595a6f7819d17faf339'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:41.461099 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-d17ebe92-471e-42df-9ad4-b5cf2ba5c944 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4290c601aa144595a6f7819d17faf339'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:41.475058 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1469/2927] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:57:41 2022] DELETE /identity/v3/users/4290c601aa144595a6f7819d17faf339 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:57:41.479857 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b127fe-6670-4043-8268-aa314f90cbc4 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:41.480145 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b127fe-6670-4043-8268-aa314f90cbc4 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:41.480418 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b127fe-6670-4043-8268-aa314f90cbc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:41.500692 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-f2b127fe-6670-4043-8268-aa314f90cbc4 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:57:41.501922 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-f2b127fe-6670-4043-8268-aa314f90cbc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:41.562209 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1459/2928] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:57:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:57:41.572156 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17ebe92-471e-42df-9ad4-b5cf2ba5c944 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:41.573598 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-d17ebe92-471e-42df-9ad4-b5cf2ba5c944 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:41.632347 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17ebe92-471e-42df-9ad4-b5cf2ba5c944 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:41.636098 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9d84a4-1a41-45c0-93e9-654104d5f407 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:41.636718 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9d84a4-1a41-45c0-93e9-654104d5f407 service neutron] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:41.636999 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9d84a4-1a41-45c0-93e9-654104d5f407 service neutron] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:41.637274 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9d84a4-1a41-45c0-93e9-654104d5f407 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:41.647454 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9d84a4-1a41-45c0-93e9-654104d5f407 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:41.649541 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9d84a4-1a41-45c0-93e9-654104d5f407 service neutron] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:41.660021 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1470/2929] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:57:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:57:41.935593 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b127fe-6670-4043-8268-aa314f90cbc4 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:41.937543 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-f2b127fe-6670-4043-8268-aa314f90cbc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:41.950078 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b127fe-6670-4043-8268-aa314f90cbc4 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:41.955284 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd47f623-31cc-457b-8284-6bb6fcc8baef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:41.955915 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd47f623-31cc-457b-8284-6bb6fcc8baef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:41.956198 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd47f623-31cc-457b-8284-6bb6fcc8baef admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:41.956475 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd47f623-31cc-457b-8284-6bb6fcc8baef admin admin] PATH_INFO: `/v3/projects/552d8c6a623645258615e961b076e5c5` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:41.957623 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd47f623-31cc-457b-8284-6bb6fcc8baef admin admin] RBAC: Authorizing `identity:delete_project(project_id=552d8c6a623645258615e961b076e5c5)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:41.959549 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd47f623-31cc-457b-8284-6bb6fcc8baef admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:42.013104 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-dd47f623-31cc-457b-8284-6bb6fcc8baef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '552d8c6a623645258615e961b076e5c5', 'request_id': 'req-dd47f623-31cc-457b-8284-6bb6fcc8baef'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:42.020105 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-dd47f623-31cc-457b-8284-6bb6fcc8baef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '552d8c6a623645258615e961b076e5c5', 'request_id': 'req-dd47f623-31cc-457b-8284-6bb6fcc8baef'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:42.025972 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-dd47f623-31cc-457b-8284-6bb6fcc8baef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '552d8c6a623645258615e961b076e5c5', 'request_id': 'req-dd47f623-31cc-457b-8284-6bb6fcc8baef'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:42.039373 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-dd47f623-31cc-457b-8284-6bb6fcc8baef admin admin] The token cache is being invalidate because project 552d8c6a623645258615e961b076e5c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81618) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:57:42.039667 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-dd47f623-31cc-457b-8284-6bb6fcc8baef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:42.040311 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-dd47f623-31cc-457b-8284-6bb6fcc8baef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:42.041797 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1460/2930] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:57:41 2022] DELETE /identity/v3/projects/552d8c6a623645258615e961b076e5c5 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:57:42.045955 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9d84a4-1a41-45c0-93e9-654104d5f407 service neutron] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:42.056040 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9d84a4-1a41-45c0-93e9-654104d5f407 service neutron] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:42.127932 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2444ca-2a7c-4663-aae1-773d7a809857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:42.128684 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2444ca-2a7c-4663-aae1-773d7a809857 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:42.128841 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2444ca-2a7c-4663-aae1-773d7a809857 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:42.129107 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2444ca-2a7c-4663-aae1-773d7a809857 admin admin] PATH_INFO: `/v3/users/062861f8618c4fbdbee4d4bda7783481` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:42.130234 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2444ca-2a7c-4663-aae1-773d7a809857 admin admin] RBAC: Authorizing `identity:delete_user(user_id=062861f8618c4fbdbee4d4bda7783481)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:42.132166 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2444ca-2a7c-4663-aae1-773d7a809857 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:42.188090 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-6f2444ca-2a7c-4663-aae1-773d7a809857 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '062861f8618c4fbdbee4d4bda7783481'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:42.188806 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-6f2444ca-2a7c-4663-aae1-773d7a809857 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '062861f8618c4fbdbee4d4bda7783481'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:42.196358 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-6f2444ca-2a7c-4663-aae1-773d7a809857 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '062861f8618c4fbdbee4d4bda7783481'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:42.208849 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:57:42.212550 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-6f2444ca-2a7c-4663-aae1-773d7a809857 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '062861f8618c4fbdbee4d4bda7783481'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:42.213442 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-6f2444ca-2a7c-4663-aae1-773d7a809857 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '062861f8618c4fbdbee4d4bda7783481'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:42.233145 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1471/2931] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:57:42 2022] DELETE /identity/v3/users/062861f8618c4fbdbee4d4bda7783481 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:57:42.533100 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd47f623-31cc-457b-8284-6bb6fcc8baef admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:42.534739 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-dd47f623-31cc-457b-8284-6bb6fcc8baef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:42.558330 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd47f623-31cc-457b-8284-6bb6fcc8baef admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:42.571730 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:57:42.631577 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a71a600-a1eb-4231-bf7f-1cd84e37c261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:42.632216 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a71a600-a1eb-4231-bf7f-1cd84e37c261 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:42.632493 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a71a600-a1eb-4231-bf7f-1cd84e37c261 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:42.632764 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a71a600-a1eb-4231-bf7f-1cd84e37c261 admin admin] PATH_INFO: `/v3/projects/f4869a6405554338a6fc246699bde0ff` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:42.633893 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a71a600-a1eb-4231-bf7f-1cd84e37c261 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4869a6405554338a6fc246699bde0ff)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:42.635824 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a71a600-a1eb-4231-bf7f-1cd84e37c261 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:42.687064 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-4a71a600-a1eb-4231-bf7f-1cd84e37c261 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4869a6405554338a6fc246699bde0ff', 'request_id': 'req-4a71a600-a1eb-4231-bf7f-1cd84e37c261'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:42.693227 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-4a71a600-a1eb-4231-bf7f-1cd84e37c261 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4869a6405554338a6fc246699bde0ff', 'request_id': 'req-4a71a600-a1eb-4231-bf7f-1cd84e37c261'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:42.698749 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-4a71a600-a1eb-4231-bf7f-1cd84e37c261 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4869a6405554338a6fc246699bde0ff', 'request_id': 'req-4a71a600-a1eb-4231-bf7f-1cd84e37c261'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:42.711822 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-4a71a600-a1eb-4231-bf7f-1cd84e37c261 admin admin] The token cache is being invalidate because project f4869a6405554338a6fc246699bde0ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81618) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:57:42.712113 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-4a71a600-a1eb-4231-bf7f-1cd84e37c261 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:42.712734 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-4a71a600-a1eb-4231-bf7f-1cd84e37c261 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:42.714250 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1461/2932] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:57:42 2022] DELETE /identity/v3/projects/f4869a6405554338a6fc246699bde0ff => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:57:42.719261 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1012e8-d23e-46f9-a28a-e5e993a79d85 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:42.719550 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1012e8-d23e-46f9-a28a-e5e993a79d85 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:42.719826 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1012e8-d23e-46f9-a28a-e5e993a79d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:42.740312 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-6f1012e8-d23e-46f9-a28a-e5e993a79d85 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:57:42.741522 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-6f1012e8-d23e-46f9-a28a-e5e993a79d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:42.804207 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1472/2933] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:57:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:57:42.812833 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5bb34a-47d4-434f-8d93-310625fa9192 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:42.813057 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5bb34a-47d4-434f-8d93-310625fa9192 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:42.813330 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5bb34a-47d4-434f-8d93-310625fa9192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:42.833529 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-7e5bb34a-47d4-434f-8d93-310625fa9192 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:57:42.834777 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-7e5bb34a-47d4-434f-8d93-310625fa9192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:42.842631 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1462/2934] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:57:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:57:42.846985 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1012e8-d23e-46f9-a28a-e5e993a79d85 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:42.856814 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1012e8-d23e-46f9-a28a-e5e993a79d85 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:42.861945 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e0d066-f55c-4894-bc9c-d3ef8261544e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:42.862578 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e0d066-f55c-4894-bc9c-d3ef8261544e admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:42.862860 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e0d066-f55c-4894-bc9c-d3ef8261544e admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:42.863133 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e0d066-f55c-4894-bc9c-d3ef8261544e admin admin] PATH_INFO: `/v3/domains` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:42.863780 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e0d066-f55c-4894-bc9c-d3ef8261544e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81617) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:57:42.864084 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e0d066-f55c-4894-bc9c-d3ef8261544e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:42.865676 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e0d066-f55c-4894-bc9c-d3ef8261544e admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:42.877708 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1473/2935] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:57:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:57:42.881913 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5bb34a-47d4-434f-8d93-310625fa9192 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:42.891132 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5bb34a-47d4-434f-8d93-310625fa9192 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:42.895802 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b866c457-111d-411e-b49d-27b326d946b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:42.896447 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b866c457-111d-411e-b49d-27b326d946b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:42.896676 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b866c457-111d-411e-b49d-27b326d946b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:42.896949 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b866c457-111d-411e-b49d-27b326d946b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:42.897584 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b866c457-111d-411e-b49d-27b326d946b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:42.899507 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b866c457-111d-411e-b49d-27b326d946b9 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:42.929799 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1463/2936] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:57:42 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:57:42.934618 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e0d066-f55c-4894-bc9c-d3ef8261544e admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:42.950221 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e0d066-f55c-4894-bc9c-d3ef8261544e admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:43.009835 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a24166-14d2-4dde-95c8-ce37883fe301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:43.010468 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a24166-14d2-4dde-95c8-ce37883fe301 admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:43.010750 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a24166-14d2-4dde-95c8-ce37883fe301 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:43.011024 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a24166-14d2-4dde-95c8-ce37883fe301 admin admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:43.011668 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a24166-14d2-4dde-95c8-ce37883fe301 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:43.013570 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a24166-14d2-4dde-95c8-ce37883fe301 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:43.034137 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1474/2937] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:57:42 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:57:43.038457 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b866c457-111d-411e-b49d-27b326d946b9 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:43.049301 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b866c457-111d-411e-b49d-27b326d946b9 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:43.053895 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3445a054-0525-4007-bff9-054cded5bbf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:43.054512 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3445a054-0525-4007-bff9-054cded5bbf4 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:43.054795 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3445a054-0525-4007-bff9-054cded5bbf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:43.055066 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3445a054-0525-4007-bff9-054cded5bbf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:43.055705 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3445a054-0525-4007-bff9-054cded5bbf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:43.057446 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3445a054-0525-4007-bff9-054cded5bbf4 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:43.068426 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1464/2938] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:57:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:57:43.072438 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a24166-14d2-4dde-95c8-ce37883fe301 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:43.082516 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a24166-14d2-4dde-95c8-ce37883fe301 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:43.087117 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975387cd-f069-4650-a9ba-91f8966f7144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:43.087707 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975387cd-f069-4650-a9ba-91f8966f7144 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:43.087981 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975387cd-f069-4650-a9ba-91f8966f7144 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:43.088250 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975387cd-f069-4650-a9ba-91f8966f7144 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:43.088881 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975387cd-f069-4650-a9ba-91f8966f7144 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:43.090482 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975387cd-f069-4650-a9ba-91f8966f7144 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:43.102086 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1475/2939] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:57:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:57:43.106352 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3445a054-0525-4007-bff9-054cded5bbf4 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:43.116997 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3445a054-0525-4007-bff9-054cded5bbf4 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:43.121694 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858d2a61-b09e-4ded-9ede-f7434995f91a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:43.122339 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858d2a61-b09e-4ded-9ede-f7434995f91a admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:43.122655 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858d2a61-b09e-4ded-9ede-f7434995f91a admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:43.123104 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858d2a61-b09e-4ded-9ede-f7434995f91a admin admin] PATH_INFO: `/v3/projects/5b1f072fae1e48d18c59bcc0d95ba89b/users/ddbbb1d00961402e9d57cd7860802aa2/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:43.138703 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858d2a61-b09e-4ded-9ede-f7434995f91a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b1f072fae1e48d18c59bcc0d95ba89b, user_id=ddbbb1d00961402e9d57cd7860802aa2, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:43.141412 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858d2a61-b09e-4ded-9ede-f7434995f91a admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:43.155968 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1465/2940] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:57:43 2022] PUT /identity/v3/projects/5b1f072fae1e48d18c59bcc0d95ba89b/users/ddbbb1d00961402e9d57cd7860802aa2/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:57:43.167184 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f78f8a-07e0-4ddd-86ea-c85d685e94b3 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:43.167466 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f78f8a-07e0-4ddd-86ea-c85d685e94b3 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:43.167738 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f78f8a-07e0-4ddd-86ea-c85d685e94b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:43.196916 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-88f78f8a-07e0-4ddd-86ea-c85d685e94b3 None None] MFA Rules not processed for user `ddbbb1d00961402e9d57cd7860802aa2`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:57:43.198131 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-88f78f8a-07e0-4ddd-86ea-c85d685e94b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:43.239876 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:57:43.265643 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1476/2941] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:57:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 16:57:43.270584 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858d2a61-b09e-4ded-9ede-f7434995f91a admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:43.315595 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:57:43.335027 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858d2a61-b09e-4ded-9ede-f7434995f91a admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:43.339985 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924d7007-6798-47e8-b448-34c56bc1bdd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:43.340587 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d7007-6798-47e8-b448-34c56bc1bdd5 admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:43.340867 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d7007-6798-47e8-b448-34c56bc1bdd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:43.341142 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d7007-6798-47e8-b448-34c56bc1bdd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:43.341790 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924d7007-6798-47e8-b448-34c56bc1bdd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:43.344246 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924d7007-6798-47e8-b448-34c56bc1bdd5 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:43.370720 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1466/2942] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:57:43 2022] POST /identity/v3/projects => generated 387 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:57:43.374804 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f78f8a-07e0-4ddd-86ea-c85d685e94b3 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:43.385668 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f78f8a-07e0-4ddd-86ea-c85d685e94b3 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:43.441270 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7794810-2006-4f7f-8b7e-2f477bc2edf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:43.441759 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7794810-2006-4f7f-8b7e-2f477bc2edf8 admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:43.442037 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7794810-2006-4f7f-8b7e-2f477bc2edf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:43.442302 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7794810-2006-4f7f-8b7e-2f477bc2edf8 admin admin] PATH_INFO: `/v3/users` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:43.442967 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7794810-2006-4f7f-8b7e-2f477bc2edf8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:43.444853 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7794810-2006-4f7f-8b7e-2f477bc2edf8 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:43.466495 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1477/2943] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:57:43 2022] POST /identity/v3/users => generated 300 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:57:43.471187 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924d7007-6798-47e8-b448-34c56bc1bdd5 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:43.481302 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924d7007-6798-47e8-b448-34c56bc1bdd5 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:43.486537 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02536b7d-41ac-4775-9cdd-40aa06a0d9fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:43.487128 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02536b7d-41ac-4775-9cdd-40aa06a0d9fa admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:43.487405 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02536b7d-41ac-4775-9cdd-40aa06a0d9fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:43.487675 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02536b7d-41ac-4775-9cdd-40aa06a0d9fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:43.488310 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02536b7d-41ac-4775-9cdd-40aa06a0d9fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:43.489872 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02536b7d-41ac-4775-9cdd-40aa06a0d9fa admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:43.500300 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1467/2944] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:57:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:57:43.504518 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7794810-2006-4f7f-8b7e-2f477bc2edf8 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:43.514419 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7794810-2006-4f7f-8b7e-2f477bc2edf8 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:43.519018 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a647c9e6-6885-499a-928b-545ecabcc925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:43.519658 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a647c9e6-6885-499a-928b-545ecabcc925 admin admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:43.519966 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a647c9e6-6885-499a-928b-545ecabcc925 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:43.520256 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a647c9e6-6885-499a-928b-545ecabcc925 admin admin] PATH_INFO: `/v3/projects/d5c84acb446a4cdb8fd7f594c6fe9ff8/users/51f68bdf08bf46d39b13906962745216/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:43.535316 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a647c9e6-6885-499a-928b-545ecabcc925 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5c84acb446a4cdb8fd7f594c6fe9ff8, user_id=51f68bdf08bf46d39b13906962745216, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:43.537937 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a647c9e6-6885-499a-928b-545ecabcc925 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:43.551346 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1478/2945] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:57:43 2022] PUT /identity/v3/projects/d5c84acb446a4cdb8fd7f594c6fe9ff8/users/51f68bdf08bf46d39b13906962745216/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 16:57:43.555277 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02536b7d-41ac-4775-9cdd-40aa06a0d9fa admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:43.565731 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02536b7d-41ac-4775-9cdd-40aa06a0d9fa admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:43.630052 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb626df-91d1-445f-aa1b-e8e040d71d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:43.630612 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb626df-91d1-445f-aa1b-e8e040d71d94 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:43.630887 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb626df-91d1-445f-aa1b-e8e040d71d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:43.631155 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb626df-91d1-445f-aa1b-e8e040d71d94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:43.631791 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb626df-91d1-445f-aa1b-e8e040d71d94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:43.633385 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb626df-91d1-445f-aa1b-e8e040d71d94 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:43.643926 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1468/2946] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:57:43 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 16:57:43.648448 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a647c9e6-6885-499a-928b-545ecabcc925 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:43.659898 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a647c9e6-6885-499a-928b-545ecabcc925 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:43.664575 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb1c016-857d-4c4c-8049-bc591b6b96a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:43.665228 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb1c016-857d-4c4c-8049-bc591b6b96a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:43.665510 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb1c016-857d-4c4c-8049-bc591b6b96a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:43.665778 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb1c016-857d-4c4c-8049-bc591b6b96a8 admin admin] PATH_INFO: `/v3/projects/d5c84acb446a4cdb8fd7f594c6fe9ff8/users/51f68bdf08bf46d39b13906962745216/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:43.667751 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb1c016-857d-4c4c-8049-bc591b6b96a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5c84acb446a4cdb8fd7f594c6fe9ff8, user_id=51f68bdf08bf46d39b13906962745216, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:43.670401 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb1c016-857d-4c4c-8049-bc591b6b96a8 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:43.681281 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1479/2947] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:57:43 2022] PUT /identity/v3/projects/d5c84acb446a4cdb8fd7f594c6fe9ff8/users/51f68bdf08bf46d39b13906962745216/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:57:43.691854 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072eab74-c756-423f-ae78-bda399261af0 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:43.692116 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072eab74-c756-423f-ae78-bda399261af0 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:43.692394 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072eab74-c756-423f-ae78-bda399261af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:43.721663 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-072eab74-c756-423f-ae78-bda399261af0 None None] MFA Rules not processed for user `51f68bdf08bf46d39b13906962745216`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:57:43.722953 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-072eab74-c756-423f-ae78-bda399261af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:43.794265 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1469/2948] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:57:43 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:57:43.806612 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb1c016-857d-4c4c-8049-bc591b6b96a8 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:43.808373 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-2cb1c016-857d-4c4c-8049-bc591b6b96a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:43.867232 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb1c016-857d-4c4c-8049-bc591b6b96a8 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:43.870902 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c63f2c9-8723-4f08-84fb-93c9cbf34d98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:43.871507 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c63f2c9-8723-4f08-84fb-93c9cbf34d98 service cinder] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:43.871785 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c63f2c9-8723-4f08-84fb-93c9cbf34d98 service cinder] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:43.872058 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c63f2c9-8723-4f08-84fb-93c9cbf34d98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:43.881953 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c63f2c9-8723-4f08-84fb-93c9cbf34d98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:43.883349 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c63f2c9-8723-4f08-84fb-93c9cbf34d98 service cinder] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:43.894053 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1480/2949] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:57:43 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:57:43.976615 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072eab74-c756-423f-ae78-bda399261af0 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:44.030189 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:57:44.052824 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072eab74-c756-423f-ae78-bda399261af0 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:44.057652 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d74a75-faa6-49f6-9b4b-bd2f9b45a7ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:44.058286 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d74a75-faa6-49f6-9b4b-bd2f9b45a7ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:44.058621 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d74a75-faa6-49f6-9b4b-bd2f9b45a7ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:44.058900 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d74a75-faa6-49f6-9b4b-bd2f9b45a7ed admin admin] PATH_INFO: `/v3/users/ddbbb1d00961402e9d57cd7860802aa2` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:44.060333 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d74a75-faa6-49f6-9b4b-bd2f9b45a7ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=ddbbb1d00961402e9d57cd7860802aa2)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:44.062772 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d74a75-faa6-49f6-9b4b-bd2f9b45a7ed admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:44.111608 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-79d74a75-faa6-49f6-9b4b-bd2f9b45a7ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ddbbb1d00961402e9d57cd7860802aa2'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:44.118862 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-79d74a75-faa6-49f6-9b4b-bd2f9b45a7ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ddbbb1d00961402e9d57cd7860802aa2'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:44.131936 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-79d74a75-faa6-49f6-9b4b-bd2f9b45a7ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ddbbb1d00961402e9d57cd7860802aa2'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:44.140629 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-79d74a75-faa6-49f6-9b4b-bd2f9b45a7ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ddbbb1d00961402e9d57cd7860802aa2'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:44.141336 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-79d74a75-faa6-49f6-9b4b-bd2f9b45a7ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ddbbb1d00961402e9d57cd7860802aa2'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:44.147586 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1470/2950] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:57:43 2022] DELETE /identity/v3/users/ddbbb1d00961402e9d57cd7860802aa2 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:57:44.152180 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4880aa3-647a-4a99-a726-8c028736dd45 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:44.152665 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4880aa3-647a-4a99-a726-8c028736dd45 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:44.152859 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4880aa3-647a-4a99-a726-8c028736dd45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:44.171942 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-a4880aa3-647a-4a99-a726-8c028736dd45 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:57:44.173161 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-a4880aa3-647a-4a99-a726-8c028736dd45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:44.206573 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:57:44.234538 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1481/2951] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:57:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:57:44.244271 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d74a75-faa6-49f6-9b4b-bd2f9b45a7ed admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:44.245689 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-79d74a75-faa6-49f6-9b4b-bd2f9b45a7ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:44.302846 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d74a75-faa6-49f6-9b4b-bd2f9b45a7ed admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:44.306553 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dff9dd2-f48f-4130-af2d-cee586826049 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:44.307156 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff9dd2-f48f-4130-af2d-cee586826049 service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:44.307434 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff9dd2-f48f-4130-af2d-cee586826049 service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:44.307705 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff9dd2-f48f-4130-af2d-cee586826049 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:44.317680 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dff9dd2-f48f-4130-af2d-cee586826049 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:44.319165 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dff9dd2-f48f-4130-af2d-cee586826049 service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:44.330038 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1471/2952] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:57:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:57:44.609059 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4880aa3-647a-4a99-a726-8c028736dd45 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:44.610503 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-a4880aa3-647a-4a99-a726-8c028736dd45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:44.623968 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4880aa3-647a-4a99-a726-8c028736dd45 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:44.628963 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce59071-6006-495a-b444-61b19242e200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:44.629606 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce59071-6006-495a-b444-61b19242e200 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:44.629889 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce59071-6006-495a-b444-61b19242e200 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:44.630167 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce59071-6006-495a-b444-61b19242e200 admin admin] PATH_INFO: `/v3/projects/5b1f072fae1e48d18c59bcc0d95ba89b` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:44.631342 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce59071-6006-495a-b444-61b19242e200 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b1f072fae1e48d18c59bcc0d95ba89b)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:44.633361 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce59071-6006-495a-b444-61b19242e200 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:44.682087 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-cce59071-6006-495a-b444-61b19242e200 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b1f072fae1e48d18c59bcc0d95ba89b', 'request_id': 'req-cce59071-6006-495a-b444-61b19242e200'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:44.689125 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-cce59071-6006-495a-b444-61b19242e200 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b1f072fae1e48d18c59bcc0d95ba89b', 'request_id': 'req-cce59071-6006-495a-b444-61b19242e200'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:44.694920 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-cce59071-6006-495a-b444-61b19242e200 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b1f072fae1e48d18c59bcc0d95ba89b', 'request_id': 'req-cce59071-6006-495a-b444-61b19242e200'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:44.707842 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-cce59071-6006-495a-b444-61b19242e200 admin admin] The token cache is being invalidate because project 5b1f072fae1e48d18c59bcc0d95ba89b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81617) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:57:44.708132 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-cce59071-6006-495a-b444-61b19242e200 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:44.708822 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-cce59071-6006-495a-b444-61b19242e200 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:44.710322 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1482/2953] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:57:44 2022] DELETE /identity/v3/projects/5b1f072fae1e48d18c59bcc0d95ba89b => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:57:44.714446 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dff9dd2-f48f-4130-af2d-cee586826049 service neutron] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:44.724791 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dff9dd2-f48f-4130-af2d-cee586826049 service neutron] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:44.784894 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750f826a-7bcc-40d9-b772-5858f152be3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:44.785903 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750f826a-7bcc-40d9-b772-5858f152be3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:44.786423 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750f826a-7bcc-40d9-b772-5858f152be3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:44.786843 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750f826a-7bcc-40d9-b772-5858f152be3b admin admin] PATH_INFO: `/v3/users/51f68bdf08bf46d39b13906962745216` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:44.788407 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750f826a-7bcc-40d9-b772-5858f152be3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=51f68bdf08bf46d39b13906962745216)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:44.790647 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750f826a-7bcc-40d9-b772-5858f152be3b admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:44.792350 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:57:44.840625 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-750f826a-7bcc-40d9-b772-5858f152be3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51f68bdf08bf46d39b13906962745216'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:44.847919 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-750f826a-7bcc-40d9-b772-5858f152be3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51f68bdf08bf46d39b13906962745216'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:44.860486 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-750f826a-7bcc-40d9-b772-5858f152be3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51f68bdf08bf46d39b13906962745216'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:44.868556 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-750f826a-7bcc-40d9-b772-5858f152be3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51f68bdf08bf46d39b13906962745216'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:44.869169 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-750f826a-7bcc-40d9-b772-5858f152be3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51f68bdf08bf46d39b13906962745216'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:44.875366 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1472/2954] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:57:44 2022] DELETE /identity/v3/users/51f68bdf08bf46d39b13906962745216 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:57:45.171902 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce59071-6006-495a-b444-61b19242e200 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:45.173367 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-cce59071-6006-495a-b444-61b19242e200 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:45.188371 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce59071-6006-495a-b444-61b19242e200 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:45.253289 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ebf06d-3bf1-4992-9c77-98d8ae3c3d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:45.253933 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ebf06d-3bf1-4992-9c77-98d8ae3c3d7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:45.254214 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ebf06d-3bf1-4992-9c77-98d8ae3c3d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:45.254530 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ebf06d-3bf1-4992-9c77-98d8ae3c3d7f admin admin] PATH_INFO: `/v3/projects/d5c84acb446a4cdb8fd7f594c6fe9ff8` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:45.255692 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ebf06d-3bf1-4992-9c77-98d8ae3c3d7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5c84acb446a4cdb8fd7f594c6fe9ff8)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:45.257654 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ebf06d-3bf1-4992-9c77-98d8ae3c3d7f admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:45.308930 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-f9ebf06d-3bf1-4992-9c77-98d8ae3c3d7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5c84acb446a4cdb8fd7f594c6fe9ff8', 'request_id': 'req-f9ebf06d-3bf1-4992-9c77-98d8ae3c3d7f'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:45.317658 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-f9ebf06d-3bf1-4992-9c77-98d8ae3c3d7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5c84acb446a4cdb8fd7f594c6fe9ff8', 'request_id': 'req-f9ebf06d-3bf1-4992-9c77-98d8ae3c3d7f'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:45.324046 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-f9ebf06d-3bf1-4992-9c77-98d8ae3c3d7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5c84acb446a4cdb8fd7f594c6fe9ff8', 'request_id': 'req-f9ebf06d-3bf1-4992-9c77-98d8ae3c3d7f'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:45.325683 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:57:45.340066 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-f9ebf06d-3bf1-4992-9c77-98d8ae3c3d7f admin admin] The token cache is being invalidate because project d5c84acb446a4cdb8fd7f594c6fe9ff8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81617) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:57:45.340364 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-f9ebf06d-3bf1-4992-9c77-98d8ae3c3d7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:45.341126 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-f9ebf06d-3bf1-4992-9c77-98d8ae3c3d7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:45.343157 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1483/2955] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:57:45 2022] DELETE /identity/v3/projects/d5c84acb446a4cdb8fd7f594c6fe9ff8 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:57:45.348147 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80fd0d7-eead-47df-987b-bb6983f1a3c3 None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:45.348445 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80fd0d7-eead-47df-987b-bb6983f1a3c3 None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:45.348772 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80fd0d7-eead-47df-987b-bb6983f1a3c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:45.368536 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-f80fd0d7-eead-47df-987b-bb6983f1a3c3 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:57:45.369799 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-f80fd0d7-eead-47df-987b-bb6983f1a3c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:45.431628 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1473/2956] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:57:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:57:45.441812 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae110e2-1545-4bd7-8f0d-f58c4a9e0b45 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:45.442097 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae110e2-1545-4bd7-8f0d-f58c4a9e0b45 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:45.442406 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae110e2-1545-4bd7-8f0d-f58c4a9e0b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:45.463160 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-aae110e2-1545-4bd7-8f0d-f58c4a9e0b45 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:57:45.464388 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-aae110e2-1545-4bd7-8f0d-f58c4a9e0b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:45.472487 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1484/2957] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:57:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:57:45.476880 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80fd0d7-eead-47df-987b-bb6983f1a3c3 None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:45.487136 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80fd0d7-eead-47df-987b-bb6983f1a3c3 None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:45.492413 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d9c3fe-00bd-4d7d-9cf7-e249bb70acd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:45.493053 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d9c3fe-00bd-4d7d-9cf7-e249bb70acd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:45.493338 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d9c3fe-00bd-4d7d-9cf7-e249bb70acd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:45.493614 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d9c3fe-00bd-4d7d-9cf7-e249bb70acd8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:45.494270 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d9c3fe-00bd-4d7d-9cf7-e249bb70acd8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:57:45.494608 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d9c3fe-00bd-4d7d-9cf7-e249bb70acd8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:45.496230 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d9c3fe-00bd-4d7d-9cf7-e249bb70acd8 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:45.510105 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1474/2958] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:57:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:57:45.515608 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae110e2-1545-4bd7-8f0d-f58c4a9e0b45 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:45.525248 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae110e2-1545-4bd7-8f0d-f58c4a9e0b45 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:45.530968 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb1da86-627b-46be-bde2-6a50d375d4e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:45.531644 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb1da86-627b-46be-bde2-6a50d375d4e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:45.531919 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb1da86-627b-46be-bde2-6a50d375d4e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:45.532229 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb1da86-627b-46be-bde2-6a50d375d4e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:45.532983 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb1da86-627b-46be-bde2-6a50d375d4e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:45.535113 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb1da86-627b-46be-bde2-6a50d375d4e3 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:45.566727 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1485/2959] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:57:45 2022] POST /identity/v3/projects => generated 379 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:57:45.571235 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d9c3fe-00bd-4d7d-9cf7-e249bb70acd8 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:45.582165 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d9c3fe-00bd-4d7d-9cf7-e249bb70acd8 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:45.641232 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae041319-0c67-440b-8d54-ec4477568c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:45.641833 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae041319-0c67-440b-8d54-ec4477568c0b admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:45.642108 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae041319-0c67-440b-8d54-ec4477568c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:45.642399 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae041319-0c67-440b-8d54-ec4477568c0b admin admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:45.643042 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae041319-0c67-440b-8d54-ec4477568c0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:45.644943 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae041319-0c67-440b-8d54-ec4477568c0b admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:45.656499 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:57:45.669648 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1475/2960] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:57:45 2022] POST /identity/v3/users => generated 290 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:57:45.673445 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb1da86-627b-46be-bde2-6a50d375d4e3 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:45.684093 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb1da86-627b-46be-bde2-6a50d375d4e3 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:45.689543 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd00587a-603e-45fc-92bb-9c0600cd4ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:45.690152 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd00587a-603e-45fc-92bb-9c0600cd4ee2 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:45.690546 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd00587a-603e-45fc-92bb-9c0600cd4ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:45.690839 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd00587a-603e-45fc-92bb-9c0600cd4ee2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:45.691550 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd00587a-603e-45fc-92bb-9c0600cd4ee2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:45.693144 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd00587a-603e-45fc-92bb-9c0600cd4ee2 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:45.704079 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1486/2961] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:57:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:57:45.708590 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae041319-0c67-440b-8d54-ec4477568c0b admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:45.718948 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae041319-0c67-440b-8d54-ec4477568c0b admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:45.723550 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff27f2b-9a19-457f-90da-40bcf439ef00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:45.724176 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff27f2b-9a19-457f-90da-40bcf439ef00 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:45.724453 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff27f2b-9a19-457f-90da-40bcf439ef00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:45.724725 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff27f2b-9a19-457f-90da-40bcf439ef00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:45.725362 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff27f2b-9a19-457f-90da-40bcf439ef00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:45.726996 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff27f2b-9a19-457f-90da-40bcf439ef00 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:45.737952 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1476/2962] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:57:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:57:45.742211 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd00587a-603e-45fc-92bb-9c0600cd4ee2 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:45.752767 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd00587a-603e-45fc-92bb-9c0600cd4ee2 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:45.757554 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8e9bea-dcfd-4e3d-9372-4882cb39e953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:45.758239 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e9bea-dcfd-4e3d-9372-4882cb39e953 admin admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:45.760109 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e9bea-dcfd-4e3d-9372-4882cb39e953 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:45.760416 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e9bea-dcfd-4e3d-9372-4882cb39e953 admin admin] PATH_INFO: `/v3/projects/d19e119c493448a29393408bded1aa46/users/d643bcd881404a6dac24b698ac157e01/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:45.776550 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8e9bea-dcfd-4e3d-9372-4882cb39e953 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d19e119c493448a29393408bded1aa46, user_id=d643bcd881404a6dac24b698ac157e01, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:45.779332 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8e9bea-dcfd-4e3d-9372-4882cb39e953 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:45.792618 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1487/2963] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:57:45 2022] PUT /identity/v3/projects/d19e119c493448a29393408bded1aa46/users/d643bcd881404a6dac24b698ac157e01/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:57:45.803961 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac579c7-1653-434e-9dcc-0b713f307d0c None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:45.804247 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac579c7-1653-434e-9dcc-0b713f307d0c None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:45.804527 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac579c7-1653-434e-9dcc-0b713f307d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:45.833537 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-5ac579c7-1653-434e-9dcc-0b713f307d0c None None] MFA Rules not processed for user `d643bcd881404a6dac24b698ac157e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:57:45.834811 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-5ac579c7-1653-434e-9dcc-0b713f307d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:45.899583 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1477/2964] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:57:45 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:57:45.904154 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8e9bea-dcfd-4e3d-9372-4882cb39e953 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:45.968101 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8e9bea-dcfd-4e3d-9372-4882cb39e953 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:45.972881 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340444f2-f3b9-4e8d-8994-09b8ae14490e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:45.973455 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340444f2-f3b9-4e8d-8994-09b8ae14490e admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:45.973739 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340444f2-f3b9-4e8d-8994-09b8ae14490e admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:45.974045 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340444f2-f3b9-4e8d-8994-09b8ae14490e admin admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:45.974764 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340444f2-f3b9-4e8d-8994-09b8ae14490e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:45.976692 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340444f2-f3b9-4e8d-8994-09b8ae14490e admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:46.008963 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1488/2965] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:57:45 2022] POST /identity/v3/projects => generated 379 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:57:46.012386 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac579c7-1653-434e-9dcc-0b713f307d0c None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:46.023229 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac579c7-1653-434e-9dcc-0b713f307d0c None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:46.082475 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a73b27-381b-463a-8bdf-f37f1614ab8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:46.083090 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a73b27-381b-463a-8bdf-f37f1614ab8a admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:46.083382 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a73b27-381b-463a-8bdf-f37f1614ab8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:46.083658 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a73b27-381b-463a-8bdf-f37f1614ab8a admin admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:46.084304 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a73b27-381b-463a-8bdf-f37f1614ab8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:46.086214 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a73b27-381b-463a-8bdf-f37f1614ab8a admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:46.108929 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1478/2966] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:57:46 2022] POST /identity/v3/users => generated 290 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:57:46.113319 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340444f2-f3b9-4e8d-8994-09b8ae14490e admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:46.124815 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340444f2-f3b9-4e8d-8994-09b8ae14490e admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:46.129926 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77deb728-8617-493e-9a6c-ac97c7489a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:46.130576 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77deb728-8617-493e-9a6c-ac97c7489a36 admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:46.130859 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77deb728-8617-493e-9a6c-ac97c7489a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:46.131135 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77deb728-8617-493e-9a6c-ac97c7489a36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:46.131793 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77deb728-8617-493e-9a6c-ac97c7489a36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:46.133406 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77deb728-8617-493e-9a6c-ac97c7489a36 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:46.145740 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1489/2967] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:57:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:57:46.149874 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a73b27-381b-463a-8bdf-f37f1614ab8a admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:46.160543 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a73b27-381b-463a-8bdf-f37f1614ab8a admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:46.165315 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f9b4ef-47f8-4b5a-8710-d054528f745d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:46.165922 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9b4ef-47f8-4b5a-8710-d054528f745d admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:46.166202 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9b4ef-47f8-4b5a-8710-d054528f745d admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:46.166516 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9b4ef-47f8-4b5a-8710-d054528f745d admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:46.167162 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f9b4ef-47f8-4b5a-8710-d054528f745d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:46.168762 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f9b4ef-47f8-4b5a-8710-d054528f745d admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:46.179780 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1479/2968] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:57:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:57:46.184751 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77deb728-8617-493e-9a6c-ac97c7489a36 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:46.195765 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77deb728-8617-493e-9a6c-ac97c7489a36 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:46.200493 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329268ad-1dc5-4759-ac9f-4e3ff42c1fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:46.201154 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329268ad-1dc5-4759-ac9f-4e3ff42c1fc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:46.201421 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329268ad-1dc5-4759-ac9f-4e3ff42c1fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:46.201694 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329268ad-1dc5-4759-ac9f-4e3ff42c1fc1 admin admin] PATH_INFO: `/v3/projects/489782f55d4f46edb561e38afd3755c7/users/09409be20a8e4a47859e43dde5396924/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:46.219075 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329268ad-1dc5-4759-ac9f-4e3ff42c1fc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=489782f55d4f46edb561e38afd3755c7, user_id=09409be20a8e4a47859e43dde5396924, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:46.222542 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329268ad-1dc5-4759-ac9f-4e3ff42c1fc1 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:46.238213 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1490/2969] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:57:46 2022] PUT /identity/v3/projects/489782f55d4f46edb561e38afd3755c7/users/09409be20a8e4a47859e43dde5396924/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:57:46.248962 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d80ac0-16b8-49f3-9a26-7d619ae628ab None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:46.249247 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d80ac0-16b8-49f3-9a26-7d619ae628ab None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:46.249544 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d80ac0-16b8-49f3-9a26-7d619ae628ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:46.283163 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-26d80ac0-16b8-49f3-9a26-7d619ae628ab None None] MFA Rules not processed for user `09409be20a8e4a47859e43dde5396924`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:57:46.284400 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-26d80ac0-16b8-49f3-9a26-7d619ae628ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:46.328020 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:57:46.353427 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1480/2970] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:57:46 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:57:46.358747 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329268ad-1dc5-4759-ac9f-4e3ff42c1fc1 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:46.399237 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:INSERT=5,keystone:DELETE=1,keystone:SELECT=94 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:57:46.422696 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329268ad-1dc5-4759-ac9f-4e3ff42c1fc1 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:46.428285 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78fb646-8b8e-4525-8975-53dbbf35fa85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:46.428922 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78fb646-8b8e-4525-8975-53dbbf35fa85 admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:46.429207 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78fb646-8b8e-4525-8975-53dbbf35fa85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:46.429483 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78fb646-8b8e-4525-8975-53dbbf35fa85 admin admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:46.430141 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78fb646-8b8e-4525-8975-53dbbf35fa85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:46.432153 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78fb646-8b8e-4525-8975-53dbbf35fa85 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:46.462966 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1491/2971] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:57:46 2022] POST /identity/v3/projects => generated 379 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:57:46.467838 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d80ac0-16b8-49f3-9a26-7d619ae628ab None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:46.489633 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d80ac0-16b8-49f3-9a26-7d619ae628ab None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:46.578681 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db6eccf-2938-4bd6-a8d9-49dbe3e58e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:46.579288 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db6eccf-2938-4bd6-a8d9-49dbe3e58e5d admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:46.579565 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db6eccf-2938-4bd6-a8d9-49dbe3e58e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:46.579835 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db6eccf-2938-4bd6-a8d9-49dbe3e58e5d admin admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:46.580503 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db6eccf-2938-4bd6-a8d9-49dbe3e58e5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:46.582408 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db6eccf-2938-4bd6-a8d9-49dbe3e58e5d admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:46.609804 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1481/2972] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:57:46 2022] POST /identity/v3/users => generated 296 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:57:46.611278 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78fb646-8b8e-4525-8975-53dbbf35fa85 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:46.622670 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78fb646-8b8e-4525-8975-53dbbf35fa85 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:46.628067 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c929bdd8-0687-435b-9f30-ab9f637409ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:46.628669 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c929bdd8-0687-435b-9f30-ab9f637409ff admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:46.628953 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c929bdd8-0687-435b-9f30-ab9f637409ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:46.629231 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c929bdd8-0687-435b-9f30-ab9f637409ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:46.629872 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c929bdd8-0687-435b-9f30-ab9f637409ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:46.631514 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c929bdd8-0687-435b-9f30-ab9f637409ff admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:46.650182 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1492/2973] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:57:46 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:57:46.654323 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db6eccf-2938-4bd6-a8d9-49dbe3e58e5d admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:46.665305 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db6eccf-2938-4bd6-a8d9-49dbe3e58e5d admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:46.670025 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cb9ee8-c0e4-476f-8a96-3201a5d564d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:46.670769 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb9ee8-c0e4-476f-8a96-3201a5d564d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:46.671073 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb9ee8-c0e4-476f-8a96-3201a5d564d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:46.671325 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb9ee8-c0e4-476f-8a96-3201a5d564d7 admin admin] PATH_INFO: `/v3/projects/75f47fb497774c2893a031f2b8e7acff/users/1e0591e9dc364b0cae16f269c32c8e23/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:46.687353 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cb9ee8-c0e4-476f-8a96-3201a5d564d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75f47fb497774c2893a031f2b8e7acff, user_id=1e0591e9dc364b0cae16f269c32c8e23, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:46.690014 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cb9ee8-c0e4-476f-8a96-3201a5d564d7 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:46.705338 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1482/2974] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:57:46 2022] PUT /identity/v3/projects/75f47fb497774c2893a031f2b8e7acff/users/1e0591e9dc364b0cae16f269c32c8e23/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:57:46.709754 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c929bdd8-0687-435b-9f30-ab9f637409ff admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:46.720901 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c929bdd8-0687-435b-9f30-ab9f637409ff admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:46.782757 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dae3d8-a180-4179-8383-9602a27381ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:46.783374 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dae3d8-a180-4179-8383-9602a27381ef admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:46.783659 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dae3d8-a180-4179-8383-9602a27381ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:46.783941 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dae3d8-a180-4179-8383-9602a27381ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:46.784589 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dae3d8-a180-4179-8383-9602a27381ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:46.786202 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dae3d8-a180-4179-8383-9602a27381ef admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:46.796124 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1493/2975] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:57:46 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:57:46.800316 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cb9ee8-c0e4-476f-8a96-3201a5d564d7 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:46.810924 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cb9ee8-c0e4-476f-8a96-3201a5d564d7 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:46.817191 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab0928-b567-49fc-ba07-df37b53a15c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:46.818267 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab0928-b567-49fc-ba07-df37b53a15c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:46.818585 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab0928-b567-49fc-ba07-df37b53a15c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:46.818867 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab0928-b567-49fc-ba07-df37b53a15c2 admin admin] PATH_INFO: `/v3/projects/75f47fb497774c2893a031f2b8e7acff/users/1e0591e9dc364b0cae16f269c32c8e23/roles/63d2d197bfd6497583e94bfb065ce29b` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:46.821009 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eab0928-b567-49fc-ba07-df37b53a15c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75f47fb497774c2893a031f2b8e7acff, user_id=1e0591e9dc364b0cae16f269c32c8e23, role_id=63d2d197bfd6497583e94bfb065ce29b)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:46.823713 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eab0928-b567-49fc-ba07-df37b53a15c2 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:46.834978 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1483/2976] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:57:46 2022] PUT /identity/v3/projects/75f47fb497774c2893a031f2b8e7acff/users/1e0591e9dc364b0cae16f269c32c8e23/roles/63d2d197bfd6497583e94bfb065ce29b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 16:57:46.845930 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1789efb9-cd6a-48f9-a8db-4c97e6eb4721 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:46.846217 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1789efb9-cd6a-48f9-a8db-4c97e6eb4721 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:46.846537 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1789efb9-cd6a-48f9-a8db-4c97e6eb4721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:46.875041 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-1789efb9-cd6a-48f9-a8db-4c97e6eb4721 None None] MFA Rules not processed for user `1e0591e9dc364b0cae16f269c32c8e23`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:57:46.876266 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-1789efb9-cd6a-48f9-a8db-4c97e6eb4721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:46.943719 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1494/2977] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:57:46 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:57:46.955983 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab0928-b567-49fc-ba07-df37b53a15c2 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:46.957688 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-7eab0928-b567-49fc-ba07-df37b53a15c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:47.014906 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab0928-b567-49fc-ba07-df37b53a15c2 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:47.018639 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349c8f13-09c9-4ebf-a814-eceab0536021 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:47.019235 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349c8f13-09c9-4ebf-a814-eceab0536021 service cinder] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:47.019511 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349c8f13-09c9-4ebf-a814-eceab0536021 service cinder] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:47.019783 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349c8f13-09c9-4ebf-a814-eceab0536021 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:47.028516 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349c8f13-09c9-4ebf-a814-eceab0536021 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:47.030009 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349c8f13-09c9-4ebf-a814-eceab0536021 service cinder] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:47.039259 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1484/2978] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:57:46 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:57:47.083187 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1789efb9-cd6a-48f9-a8db-4c97e6eb4721 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:47.084754 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-1789efb9-cd6a-48f9-a8db-4c97e6eb4721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:47.097000 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1789efb9-cd6a-48f9-a8db-4c97e6eb4721 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:47.100689 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba38856-7935-44d5-b646-b37ea5896f8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:47.101287 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba38856-7935-44d5-b646-b37ea5896f8a service cinder] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:47.101567 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba38856-7935-44d5-b646-b37ea5896f8a service cinder] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:47.101842 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba38856-7935-44d5-b646-b37ea5896f8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:47.132674 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:57:47.160615 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba38856-7935-44d5-b646-b37ea5896f8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:47.162051 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba38856-7935-44d5-b646-b37ea5896f8a service cinder] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:47.171068 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1495/2979] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:57:47 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 16:57:50.074858 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349c8f13-09c9-4ebf-a814-eceab0536021 service cinder] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:50.128617 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:57:50.152327 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349c8f13-09c9-4ebf-a814-eceab0536021 service cinder] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:50.156928 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f21aa-abbe-47f7-a785-26fa5bbda411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:50.157552 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f21aa-abbe-47f7-a785-26fa5bbda411 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:50.157832 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f21aa-abbe-47f7-a785-26fa5bbda411 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:50.158106 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f21aa-abbe-47f7-a785-26fa5bbda411 admin admin] PATH_INFO: `/v3/users/d643bcd881404a6dac24b698ac157e01` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:50.159352 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810f21aa-abbe-47f7-a785-26fa5bbda411 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d643bcd881404a6dac24b698ac157e01)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:50.161261 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810f21aa-abbe-47f7-a785-26fa5bbda411 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:50.210202 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-810f21aa-abbe-47f7-a785-26fa5bbda411 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd643bcd881404a6dac24b698ac157e01'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:50.217274 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-810f21aa-abbe-47f7-a785-26fa5bbda411 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd643bcd881404a6dac24b698ac157e01'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:50.229156 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-810f21aa-abbe-47f7-a785-26fa5bbda411 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd643bcd881404a6dac24b698ac157e01'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:50.236854 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-810f21aa-abbe-47f7-a785-26fa5bbda411 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd643bcd881404a6dac24b698ac157e01'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:50.237320 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-810f21aa-abbe-47f7-a785-26fa5bbda411 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd643bcd881404a6dac24b698ac157e01'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:50.243281 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1485/2980] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:57:50 2022] DELETE /identity/v3/users/d643bcd881404a6dac24b698ac157e01 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:57:50.247796 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f8440e-8ecc-4300-a5b7-7308fd5cea44 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:50.248786 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f8440e-8ecc-4300-a5b7-7308fd5cea44 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:50.248891 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f8440e-8ecc-4300-a5b7-7308fd5cea44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:50.269383 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-01f8440e-8ecc-4300-a5b7-7308fd5cea44 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:57:50.270638 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-01f8440e-8ecc-4300-a5b7-7308fd5cea44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:50.336581 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1496/2981] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:57:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:57:50.347414 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f21aa-abbe-47f7-a785-26fa5bbda411 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:50.348927 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-810f21aa-abbe-47f7-a785-26fa5bbda411 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:50.419333 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f21aa-abbe-47f7-a785-26fa5bbda411 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:50.423760 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bcbaba-2471-473b-b8cc-509cbc4e52ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:50.424353 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bcbaba-2471-473b-b8cc-509cbc4e52ce service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:50.424629 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bcbaba-2471-473b-b8cc-509cbc4e52ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:50.424899 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bcbaba-2471-473b-b8cc-509cbc4e52ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:50.434990 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bcbaba-2471-473b-b8cc-509cbc4e52ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:50.437674 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bcbaba-2471-473b-b8cc-509cbc4e52ce service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:50.449682 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1486/2982] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:57:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:57:50.738231 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f8440e-8ecc-4300-a5b7-7308fd5cea44 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:50.739768 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-01f8440e-8ecc-4300-a5b7-7308fd5cea44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:50.753502 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f8440e-8ecc-4300-a5b7-7308fd5cea44 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:50.761047 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de43727-8b32-46bd-8dd6-70859c3c79ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:50.761715 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de43727-8b32-46bd-8dd6-70859c3c79ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:50.762003 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de43727-8b32-46bd-8dd6-70859c3c79ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:50.762461 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de43727-8b32-46bd-8dd6-70859c3c79ea admin admin] PATH_INFO: `/v3/projects/d19e119c493448a29393408bded1aa46` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:50.763820 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de43727-8b32-46bd-8dd6-70859c3c79ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=d19e119c493448a29393408bded1aa46)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:50.766057 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de43727-8b32-46bd-8dd6-70859c3c79ea admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:50.822065 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-7de43727-8b32-46bd-8dd6-70859c3c79ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd19e119c493448a29393408bded1aa46', 'request_id': 'req-7de43727-8b32-46bd-8dd6-70859c3c79ea'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:50.828484 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-7de43727-8b32-46bd-8dd6-70859c3c79ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd19e119c493448a29393408bded1aa46', 'request_id': 'req-7de43727-8b32-46bd-8dd6-70859c3c79ea'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:50.834108 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-7de43727-8b32-46bd-8dd6-70859c3c79ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd19e119c493448a29393408bded1aa46', 'request_id': 'req-7de43727-8b32-46bd-8dd6-70859c3c79ea'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:50.850492 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-7de43727-8b32-46bd-8dd6-70859c3c79ea admin admin] The token cache is being invalidate because project d19e119c493448a29393408bded1aa46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81617) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:57:50.850689 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-7de43727-8b32-46bd-8dd6-70859c3c79ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:50.851329 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-7de43727-8b32-46bd-8dd6-70859c3c79ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:50.852825 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1497/2983] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:57:50 2022] DELETE /identity/v3/projects/d19e119c493448a29393408bded1aa46 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:57:50.857051 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bcbaba-2471-473b-b8cc-509cbc4e52ce service neutron] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:50.870796 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bcbaba-2471-473b-b8cc-509cbc4e52ce service neutron] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:50.928949 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc4aed4-d6fa-4773-b799-ad29519a8f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:50.929591 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc4aed4-d6fa-4773-b799-ad29519a8f7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:50.929873 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc4aed4-d6fa-4773-b799-ad29519a8f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:50.930145 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc4aed4-d6fa-4773-b799-ad29519a8f7e admin admin] PATH_INFO: `/v3/users/09409be20a8e4a47859e43dde5396924` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:50.931329 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc4aed4-d6fa-4773-b799-ad29519a8f7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=09409be20a8e4a47859e43dde5396924)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:50.933283 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc4aed4-d6fa-4773-b799-ad29519a8f7e admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:50.935262 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:57:50.985291 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-1fc4aed4-d6fa-4773-b799-ad29519a8f7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09409be20a8e4a47859e43dde5396924'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:50.992344 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-1fc4aed4-d6fa-4773-b799-ad29519a8f7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09409be20a8e4a47859e43dde5396924'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:51.006781 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-1fc4aed4-d6fa-4773-b799-ad29519a8f7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09409be20a8e4a47859e43dde5396924'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:51.016430 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-1fc4aed4-d6fa-4773-b799-ad29519a8f7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09409be20a8e4a47859e43dde5396924'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:51.017211 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-1fc4aed4-d6fa-4773-b799-ad29519a8f7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09409be20a8e4a47859e43dde5396924'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:51.024545 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1487/2984] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:57:50 2022] DELETE /identity/v3/users/09409be20a8e4a47859e43dde5396924 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 16:57:51.308715 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de43727-8b32-46bd-8dd6-70859c3c79ea admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:51.310476 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-7de43727-8b32-46bd-8dd6-70859c3c79ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:51.325793 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de43727-8b32-46bd-8dd6-70859c3c79ea admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:51.382220 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:57:51.391275 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b75bc18-75dc-4b73-917a-db5e816f4a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:51.392195 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b75bc18-75dc-4b73-917a-db5e816f4a67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:51.392535 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b75bc18-75dc-4b73-917a-db5e816f4a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:51.392983 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b75bc18-75dc-4b73-917a-db5e816f4a67 admin admin] PATH_INFO: `/v3/projects/489782f55d4f46edb561e38afd3755c7` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:51.394273 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b75bc18-75dc-4b73-917a-db5e816f4a67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=489782f55d4f46edb561e38afd3755c7)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:51.396282 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b75bc18-75dc-4b73-917a-db5e816f4a67 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:51.444652 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-9b75bc18-75dc-4b73-917a-db5e816f4a67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '489782f55d4f46edb561e38afd3755c7', 'request_id': 'req-9b75bc18-75dc-4b73-917a-db5e816f4a67'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:51.450740 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-9b75bc18-75dc-4b73-917a-db5e816f4a67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '489782f55d4f46edb561e38afd3755c7', 'request_id': 'req-9b75bc18-75dc-4b73-917a-db5e816f4a67'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:51.456399 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-9b75bc18-75dc-4b73-917a-db5e816f4a67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '489782f55d4f46edb561e38afd3755c7', 'request_id': 'req-9b75bc18-75dc-4b73-917a-db5e816f4a67'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:51.469712 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-9b75bc18-75dc-4b73-917a-db5e816f4a67 admin admin] The token cache is being invalidate because project 489782f55d4f46edb561e38afd3755c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81617) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:57:51.470006 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-9b75bc18-75dc-4b73-917a-db5e816f4a67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:51.470670 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-9b75bc18-75dc-4b73-917a-db5e816f4a67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:51.472163 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1498/2985] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:57:51 2022] DELETE /identity/v3/projects/489782f55d4f46edb561e38afd3755c7 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:57:51.476125 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc4aed4-d6fa-4773-b799-ad29519a8f7e admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:51.477588 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-1fc4aed4-d6fa-4773-b799-ad29519a8f7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:51.491004 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc4aed4-d6fa-4773-b799-ad29519a8f7e admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:51.549588 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9d2547-f77a-4d7b-aa35-2ecb7416ce76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:51.550227 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9d2547-f77a-4d7b-aa35-2ecb7416ce76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:51.550547 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9d2547-f77a-4d7b-aa35-2ecb7416ce76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:51.550830 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9d2547-f77a-4d7b-aa35-2ecb7416ce76 admin admin] PATH_INFO: `/v3/users/1e0591e9dc364b0cae16f269c32c8e23` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:51.551971 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9d2547-f77a-4d7b-aa35-2ecb7416ce76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e0591e9dc364b0cae16f269c32c8e23)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:51.553884 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9d2547-f77a-4d7b-aa35-2ecb7416ce76 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:51.603670 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-3d9d2547-f77a-4d7b-aa35-2ecb7416ce76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e0591e9dc364b0cae16f269c32c8e23'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:51.611307 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-3d9d2547-f77a-4d7b-aa35-2ecb7416ce76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e0591e9dc364b0cae16f269c32c8e23'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:51.623635 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-3d9d2547-f77a-4d7b-aa35-2ecb7416ce76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e0591e9dc364b0cae16f269c32c8e23'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:51.631473 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-3d9d2547-f77a-4d7b-aa35-2ecb7416ce76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e0591e9dc364b0cae16f269c32c8e23'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:51.632142 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-3d9d2547-f77a-4d7b-aa35-2ecb7416ce76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e0591e9dc364b0cae16f269c32c8e23'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:51.633707 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:57:51.639949 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1488/2986] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:57:51 2022] DELETE /identity/v3/users/1e0591e9dc364b0cae16f269c32c8e23 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:57:51.924501 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b75bc18-75dc-4b73-917a-db5e816f4a67 admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:51.926081 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-9b75bc18-75dc-4b73-917a-db5e816f4a67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:51.940433 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b75bc18-75dc-4b73-917a-db5e816f4a67 admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:52.006336 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c460dcb-7e20-498e-9da1-830c50b14220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:52.007003 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c460dcb-7e20-498e-9da1-830c50b14220 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:52.007281 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c460dcb-7e20-498e-9da1-830c50b14220 admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:52.007557 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c460dcb-7e20-498e-9da1-830c50b14220 admin admin] PATH_INFO: `/v3/projects/75f47fb497774c2893a031f2b8e7acff` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:52.008736 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c460dcb-7e20-498e-9da1-830c50b14220 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75f47fb497774c2893a031f2b8e7acff)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:52.010670 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c460dcb-7e20-498e-9da1-830c50b14220 admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:52.058916 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-9c460dcb-7e20-498e-9da1-830c50b14220 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75f47fb497774c2893a031f2b8e7acff', 'request_id': 'req-9c460dcb-7e20-498e-9da1-830c50b14220'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:52.065801 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-9c460dcb-7e20-498e-9da1-830c50b14220 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75f47fb497774c2893a031f2b8e7acff', 'request_id': 'req-9c460dcb-7e20-498e-9da1-830c50b14220'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:52.075843 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-9c460dcb-7e20-498e-9da1-830c50b14220 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75f47fb497774c2893a031f2b8e7acff', 'request_id': 'req-9c460dcb-7e20-498e-9da1-830c50b14220'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:52.089937 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-9c460dcb-7e20-498e-9da1-830c50b14220 admin admin] The token cache is being invalidate because project 75f47fb497774c2893a031f2b8e7acff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81617) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:57:52.090233 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-9c460dcb-7e20-498e-9da1-830c50b14220 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:52.090912 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-9c460dcb-7e20-498e-9da1-830c50b14220 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:57:52.093283 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1499/2987] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:57:51 2022] DELETE /identity/v3/projects/75f47fb497774c2893a031f2b8e7acff => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:57:52.099816 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ba9df9-66aa-43fa-a656-87f69e45419b None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:52.100093 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ba9df9-66aa-43fa-a656-87f69e45419b None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:52.100446 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ba9df9-66aa-43fa-a656-87f69e45419b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:52.121099 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-61ba9df9-66aa-43fa-a656-87f69e45419b None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:57:52.122504 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-61ba9df9-66aa-43fa-a656-87f69e45419b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:52.183494 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1489/2988] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:57:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:57:52.193685 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc901d-8f6f-4f97-9550-abfcf95bfa3c None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:52.193968 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc901d-8f6f-4f97-9550-abfcf95bfa3c None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:52.194258 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc901d-8f6f-4f97-9550-abfcf95bfa3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:52.216108 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-34fc901d-8f6f-4f97-9550-abfcf95bfa3c None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:57:52.217529 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-34fc901d-8f6f-4f97-9550-abfcf95bfa3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:52.226020 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1500/2989] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:57:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:57:52.231358 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ba9df9-66aa-43fa-a656-87f69e45419b None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:52.241604 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ba9df9-66aa-43fa-a656-87f69e45419b None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:52.246456 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e238943f-8109-4329-87bd-fb4aed9f61dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:52.247077 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e238943f-8109-4329-87bd-fb4aed9f61dc admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:52.247359 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e238943f-8109-4329-87bd-fb4aed9f61dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:52.247635 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e238943f-8109-4329-87bd-fb4aed9f61dc admin admin] PATH_INFO: `/v3/domains` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:52.248818 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e238943f-8109-4329-87bd-fb4aed9f61dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81618) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 16:57:52.249127 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e238943f-8109-4329-87bd-fb4aed9f61dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:52.250781 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e238943f-8109-4329-87bd-fb4aed9f61dc admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:52.264423 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1490/2990] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 19 16:57:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 16:57:52.268665 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc901d-8f6f-4f97-9550-abfcf95bfa3c None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:52.278282 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc901d-8f6f-4f97-9550-abfcf95bfa3c None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:52.283153 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade34ac0-0700-423d-9d67-ff1d5222949b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:52.283771 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade34ac0-0700-423d-9d67-ff1d5222949b admin admin] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:52.284044 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade34ac0-0700-423d-9d67-ff1d5222949b admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:52.284310 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade34ac0-0700-423d-9d67-ff1d5222949b admin admin] PATH_INFO: `/v3/projects` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:52.284953 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade34ac0-0700-423d-9d67-ff1d5222949b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:52.286893 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade34ac0-0700-423d-9d67-ff1d5222949b admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:52.292739 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:57:52.321021 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1501/2991] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 19 16:57:52 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:57:52.325415 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e238943f-8109-4329-87bd-fb4aed9f61dc admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:52.337005 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e238943f-8109-4329-87bd-fb4aed9f61dc admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:52.403110 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea7e0db-7906-412d-83da-a9aea08f158d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:52.403727 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea7e0db-7906-412d-83da-a9aea08f158d admin admin] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:52.404007 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea7e0db-7906-412d-83da-a9aea08f158d admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:52.404324 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea7e0db-7906-412d-83da-a9aea08f158d admin admin] PATH_INFO: `/v3/users` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:52.404967 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea7e0db-7906-412d-83da-a9aea08f158d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:52.406918 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea7e0db-7906-412d-83da-a9aea08f158d admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:52.430237 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1491/2992] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 19 16:57:52 2022] POST /identity/v3/users => generated 288 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 16:57:52.434891 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade34ac0-0700-423d-9d67-ff1d5222949b admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:52.445857 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade34ac0-0700-423d-9d67-ff1d5222949b admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:52.450470 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3f74a9-15c5-4f41-acf8-5866e9144adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:52.451071 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f74a9-15c5-4f41-acf8-5866e9144adb admin admin] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:52.451937 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f74a9-15c5-4f41-acf8-5866e9144adb admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:52.452209 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f74a9-15c5-4f41-acf8-5866e9144adb admin admin] PATH_INFO: `/v3/roles` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:52.452847 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3f74a9-15c5-4f41-acf8-5866e9144adb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:52.454478 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3f74a9-15c5-4f41-acf8-5866e9144adb admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:52.464784 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1502/2993] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:57:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:57:52.469095 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea7e0db-7906-412d-83da-a9aea08f158d admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:52.480467 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea7e0db-7906-412d-83da-a9aea08f158d admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:52.484242 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3520b382-74a8-4f7b-a68a-2be060468e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:52.484838 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3520b382-74a8-4f7b-a68a-2be060468e50 admin admin] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:52.485118 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3520b382-74a8-4f7b-a68a-2be060468e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:52.485388 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3520b382-74a8-4f7b-a68a-2be060468e50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:52.486058 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3520b382-74a8-4f7b-a68a-2be060468e50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:52.487707 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3520b382-74a8-4f7b-a68a-2be060468e50 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:52.499235 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1492/2994] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 19 16:57:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 16:57:52.503695 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3f74a9-15c5-4f41-acf8-5866e9144adb admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:52.514356 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3f74a9-15c5-4f41-acf8-5866e9144adb admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:52.518999 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b36c5b-c9b8-4216-9088-4829df95aa1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:52.519643 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b36c5b-c9b8-4216-9088-4829df95aa1d admin admin] REQUEST_METHOD: `PUT` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:52.519918 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b36c5b-c9b8-4216-9088-4829df95aa1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:52.520188 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b36c5b-c9b8-4216-9088-4829df95aa1d admin admin] PATH_INFO: `/v3/projects/9c074a2f83c74487a0c539cc913f9d71/users/072f6b6849194251828637041ab09e75/roles/31b0628699ea4f98824f20e7568748cc` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:52.535784 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b36c5b-c9b8-4216-9088-4829df95aa1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c074a2f83c74487a0c539cc913f9d71, user_id=072f6b6849194251828637041ab09e75, role_id=31b0628699ea4f98824f20e7568748cc)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:52.538506 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b36c5b-c9b8-4216-9088-4829df95aa1d admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:52.553332 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1503/2995] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 19 16:57:52 2022] PUT /identity/v3/projects/9c074a2f83c74487a0c539cc913f9d71/users/072f6b6849194251828637041ab09e75/roles/31b0628699ea4f98824f20e7568748cc => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:57:52.566593 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522978f2-09eb-4545-b9dc-a7ae40ebdb8e None None] REQUEST_METHOD: `POST` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:52.566882 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522978f2-09eb-4545-b9dc-a7ae40ebdb8e None None] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:52.567152 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522978f2-09eb-4545-b9dc-a7ae40ebdb8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:52.596805 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.auth.core [None req-522978f2-09eb-4545-b9dc-a7ae40ebdb8e None None] MFA Rules not processed for user `072f6b6849194251828637041ab09e75`. Rule list: `[]` (Enabled: `True`). {{(pid=81618) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:57:52.598056 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-522978f2-09eb-4545-b9dc-a7ae40ebdb8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:52.655941 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:57:52.668211 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1493/2996] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:57:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:57:52.680440 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b36c5b-c9b8-4216-9088-4829df95aa1d admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:57:52.683304 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-07b36c5b-c9b8-4216-9088-4829df95aa1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:57:52.745533 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b36c5b-c9b8-4216-9088-4829df95aa1d admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:57:52.749826 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2827f4bd-e2e2-4093-8b56-7b16e48c6ed0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d596e43cc7446d29e2d62d9052d6091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:57:52.750465 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2827f4bd-e2e2-4093-8b56-7b16e48c6ed0 service cinder] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:57:52.750745 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2827f4bd-e2e2-4093-8b56-7b16e48c6ed0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:57:52.751022 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2827f4bd-e2e2-4093-8b56-7b16e48c6ed0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:57:52.759712 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2827f4bd-e2e2-4093-8b56-7b16e48c6ed0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:57:52.761155 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2827f4bd-e2e2-4093-8b56-7b16e48c6ed0 service cinder] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:57:52.770992 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1504/2997] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:57:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 16:58:02.669630 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=3 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:58:02.766646 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=46,keystone:INSERT=2 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:58:02.999342 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522978f2-09eb-4545-b9dc-a7ae40ebdb8e None None] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:58:03.068091 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522978f2-09eb-4545-b9dc-a7ae40ebdb8e None None] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:58:03.072747 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3f2551-c57f-4c06-8d7a-490be06f63a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:58:03.073571 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3f2551-c57f-4c06-8d7a-490be06f63a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:58:03.073850 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3f2551-c57f-4c06-8d7a-490be06f63a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:58:03.074126 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3f2551-c57f-4c06-8d7a-490be06f63a9 admin admin] PATH_INFO: `/v3/users/072f6b6849194251828637041ab09e75` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:58:03.075431 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3f2551-c57f-4c06-8d7a-490be06f63a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=072f6b6849194251828637041ab09e75)` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:58:03.078045 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3f2551-c57f-4c06-8d7a-490be06f63a9 admin admin] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:58:03.129850 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-0d3f2551-c57f-4c06-8d7a-490be06f63a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '072f6b6849194251828637041ab09e75'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:58:03.137365 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-0d3f2551-c57f-4c06-8d7a-490be06f63a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '072f6b6849194251828637041ab09e75'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:58:03.149711 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-0d3f2551-c57f-4c06-8d7a-490be06f63a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '072f6b6849194251828637041ab09e75'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:58:03.157794 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-0d3f2551-c57f-4c06-8d7a-490be06f63a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '072f6b6849194251828637041ab09e75'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:58:03.158541 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.notifications [None req-0d3f2551-c57f-4c06-8d7a-490be06f63a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '072f6b6849194251828637041ab09e75'} {{(pid=81618) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:58:03.164924 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1494/2998] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 19 16:58:02 2022] DELETE /identity/v3/users/072f6b6849194251828637041ab09e75 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:58:03.170107 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095cbb7a-3926-45de-a4fd-19b15880b158 None None] REQUEST_METHOD: `POST` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:58:03.170421 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095cbb7a-3926-45de-a4fd-19b15880b158 None None] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:58:03.170702 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095cbb7a-3926-45de-a4fd-19b15880b158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:58:03.191448 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.auth.core [None req-095cbb7a-3926-45de-a4fd-19b15880b158 None None] MFA Rules not processed for user `18d4b3afb42f46999e6cf7b500999690`. Rule list: `[]` (Enabled: `True`). {{(pid=81617) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 16:58:03.192690 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-095cbb7a-3926-45de-a4fd-19b15880b158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:58:03.255936 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1505/2999] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 19 16:58:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 16:58:03.265951 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3f2551-c57f-4c06-8d7a-490be06f63a9 admin admin] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:58:03.267453 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-0d3f2551-c57f-4c06-8d7a-490be06f63a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:58:03.328608 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3f2551-c57f-4c06-8d7a-490be06f63a9 admin admin] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:58:03.332397 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e92ff1-9cb8-49a4-89c6-7a261283430b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50355d7b1c6c4f0bbbbc7fe56efb9039', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:58:03.333073 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e92ff1-9cb8-49a4-89c6-7a261283430b service neutron] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:58:03.333355 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e92ff1-9cb8-49a4-89c6-7a261283430b service neutron] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:58:03.333629 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e92ff1-9cb8-49a4-89c6-7a261283430b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:58:03.343508 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e92ff1-9cb8-49a4-89c6-7a261283430b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:58:03.344933 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e92ff1-9cb8-49a4-89c6-7a261283430b service neutron] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:58:03.355371 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1495/3000] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:58:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 16:58:03.640776 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095cbb7a-3926-45de-a4fd-19b15880b158 None None] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:58:03.647077 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-095cbb7a-3926-45de-a4fd-19b15880b158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:58:03.672908 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095cbb7a-3926-45de-a4fd-19b15880b158 None None] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:58:03.683315 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57388d99-b52d-4dfc-a5e5-35e4d65de13f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18d4b3afb42f46999e6cf7b500999690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a804037a8734a35ad3ef5ec7e34d4c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:58:03.684780 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57388d99-b52d-4dfc-a5e5-35e4d65de13f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:58:03.685573 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57388d99-b52d-4dfc-a5e5-35e4d65de13f admin admin] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:58:03.685978 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57388d99-b52d-4dfc-a5e5-35e4d65de13f admin admin] PATH_INFO: `/v3/projects/9c074a2f83c74487a0c539cc913f9d71` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:58:03.688747 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57388d99-b52d-4dfc-a5e5-35e4d65de13f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c074a2f83c74487a0c539cc913f9d71)` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:58:03.693456 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57388d99-b52d-4dfc-a5e5-35e4d65de13f admin admin] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:58:03.771782 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-57388d99-b52d-4dfc-a5e5-35e4d65de13f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c074a2f83c74487a0c539cc913f9d71', 'request_id': 'req-57388d99-b52d-4dfc-a5e5-35e4d65de13f'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:58:03.779987 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-57388d99-b52d-4dfc-a5e5-35e4d65de13f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c074a2f83c74487a0c539cc913f9d71', 'request_id': 'req-57388d99-b52d-4dfc-a5e5-35e4d65de13f'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:58:03.786457 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-57388d99-b52d-4dfc-a5e5-35e4d65de13f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c074a2f83c74487a0c539cc913f9d71', 'request_id': 'req-57388d99-b52d-4dfc-a5e5-35e4d65de13f'} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:58:03.800731 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-57388d99-b52d-4dfc-a5e5-35e4d65de13f admin admin] The token cache is being invalidate because project 9c074a2f83c74487a0c539cc913f9d71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81617) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 16:58:03.801117 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-57388d99-b52d-4dfc-a5e5-35e4d65de13f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:58:03.801682 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.notifications [None req-57388d99-b52d-4dfc-a5e5-35e4d65de13f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81617) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 16:58:03.803244 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1506/3001] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 19 16:58:03 2022] DELETE /identity/v3/projects/9c074a2f83c74487a0c539cc913f9d71 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 16:58:12.663363 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e92ff1-9cb8-49a4-89c6-7a261283430b service neutron] Authenticating user token {{(pid=81618) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 16:58:12.666202 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-34e92ff1-9cb8-49a4-89c6-7a261283430b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:58:12.716869 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:58:12.767941 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e92ff1-9cb8-49a4-89c6-7a261283430b service neutron] Validating token access rules against request {{(pid=81618) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 16:58:12.839230 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be8a04f-6e33-41d5-814f-4959031149da service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1a985495ed48a4acc48e2e6fc7f977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81618) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 16:58:12.839853 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8a04f-6e33-41d5-814f-4959031149da service placement] REQUEST_METHOD: `GET` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 16:58:12.840140 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8a04f-6e33-41d5-814f-4959031149da service placement] SCRIPT_NAME: `/identity` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 16:58:12.840413 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8a04f-6e33-41d5-814f-4959031149da service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81618) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 16:58:12.842460 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.fernet_utils [None req-7be8a04f-6e33-41d5-814f-4959031149da service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81618) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 16:58:12.854774 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be8a04f-6e33-41d5-814f-4959031149da service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 16:58:12.856383 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be8a04f-6e33-41d5-814f-4959031149da service placement] RBAC: Authorization granted {{(pid=81618) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 16:58:12.868141 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: [pid: 81618|app: 0|req: 1496/3002] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 16:58:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 16:58:13.796666 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG dbcounter [-] [81617] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81617) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:58:22.863252 dv-cinder-858370-2-iscsi devstack@keystone.service[81618]: DEBUG dbcounter [-] [81618] Writing DB stats keystone:SELECT=48 {{(pid=81618) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 17:03:18.634001 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57388d99-b52d-4dfc-a5e5-35e4d65de13f admin admin] Authenticating user token {{(pid=81617) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 17:03:18.635501 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.fernet_utils [None req-57388d99-b52d-4dfc-a5e5-35e4d65de13f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81617) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 17:03:18.724380 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57388d99-b52d-4dfc-a5e5-35e4d65de13f admin admin] Validating token access rules against request {{(pid=81617) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 17:03:18.729975 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b21eb42-62b0-48de-b43e-0f9cf4589d5b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1a985495ed48a4acc48e2e6fc7f977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be03e5300b3343169f35c643c93d1e06', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81617) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 17:03:18.730651 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b21eb42-62b0-48de-b43e-0f9cf4589d5b service placement] REQUEST_METHOD: `GET` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 17:03:18.730928 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b21eb42-62b0-48de-b43e-0f9cf4589d5b service placement] SCRIPT_NAME: `/identity` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 17:03:18.731196 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b21eb42-62b0-48de-b43e-0f9cf4589d5b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81617) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 17:03:18.740699 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b21eb42-62b0-48de-b43e-0f9cf4589d5b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 17:03:18.742298 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b21eb42-62b0-48de-b43e-0f9cf4589d5b service placement] RBAC: Authorization granted {{(pid=81617) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 17:03:18.753136 dv-cinder-858370-2-iscsi devstack@keystone.service[81617]: [pid: 81617|app: 0|req: 1507/3003] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 19 17:03:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0)